Sophon

# Pre-requisites

Before diving into deployment of your mining operation, please make sure you go through the following steps.

WARNING

It is recommended that you test your setup in calibration network before deploying on mainnet.

# Setup your permanent storage

Choose a network file system that you are familiar with (NFS for example) and deploy your storage cluster.

# Get your account setup in Sophon service

There are two ways to have your account setup.

# For miners connecting to Sophon service

If you are trying to connect to a hosted Sophon service, contact admin of said service and have them set it up for you.

TIP

venus-wallet can be deployed as either a Sophon service or a local component depend on your security requirement.

# For admins of Sophon service

If you are an admin hosting Sophon service, use the following command to create an account for your sealer clusters.

# create user, default state=1
$ ./sophon-auth user add <ACCOUNT_NAME>
# bind miner to user
$ ./sophon-auth user miner add <ACCOUNT_NAME> <MINER_ID> 

# The returned token is what miner have to add into their config file in order to gain access to your shared modules
$ ./sophon-auth token gen --perm write <ACCOUNT_NAME>
<AUTH_TOKEN_FOR_ACCOUNT_NAME>

Update user information if necessary.

./sophon-auth user update --user=<ACCOUNT_NAME> --help

# Software dependencies

You will need to install these software dependencies (opens new window) (same as Lotus) before running venus.

# Install venus-wallet

Download and compile the source code of venus-wallet.

$ git clone https://github.com/filecoin-project/venus-wallet.git
# change directory to venus-wallet
$ cd venus-wallet
$ git checkout <RELEASE_TAG>
$ make

Run venus-wallet module in background.

$ nohup ./venus-wallet run > wallet.log 2>&1 &

TIP

Use tail -f wallet.log to monitor wallet log.

Setup a password for your venus-wallet.

$ ./venus-wallet setpwd
Password:******
Enter Password again:******
Password set successfully

WARNING

Please keep backups of your password and store them properly or you will lose contorl of your wallet.

WARNING

When restarting your wallet, manuually unlock your wallet or you wallet won't be able to sign any messages during sealing or other tasks.

$ ./venus-wallet unlock
Password:

Use --password for auto unlocking after running.

$ ./venus-wallet run --password

Check current state your wallet.

$ ./venus-wallet lock-state
wallet state: unlocked

Generate owner and worker addresses. (If you don't have an existing miner id)

$ ./venus-wallet new bls
<OWNER_ADDRESS>
$ ./venus-wallet new bls
<WORKER_ADDRESS>

TIP

If you are testing on Calibration, you have to fund all your addresses with test coins from faucets. For calibration, use this faucet (opens new window).

TIP

Use ./venus-wallet import command for importing addresses from private keys.

Change [APIRegisterHub] section of ~/.venus_wallet/config.toml using the credential you get from Sophon service admin.

[APIRegisterHub]
RegisterAPI = ["/ip4/<IP_ADDRESS_OF_VENUS_GATEWAY>/tcp/45132"]
Token = "<AUTH_TOKEN_FOR_VENUS_WALLET>"
SupportAccounts = ["<ACCOUNT_NAME>"]

WARNING

Make sure above 3 params are correctly set, or connection to Sophon service will fail.

Restart venus-wallet so that the changes takes into effect.

# grep [PID] of venus-wallet process
$ ps -ef | grep wallet
root   6704  2.3  0.0 2361236 43148 pts/2   Sl   17:33   0:18 ./venus-wallet run
root   8029  0.0  0.0 112828   952 pts/2    S+   17:46   0:00 grep --color=auto venus-wallet
# kill the process and restart
$ kill -9 [PID]
$ nohup ./venus-wallet run > wallet.log 2>&1 &

You should see logs close to the following indicating successful connection to sophon-gateway.

2021-07-12T15:14:12.457+0800    INFO    wallet_event    wallet_event/listenevent.go:197 connect to server fcf714b2-eeb6-498b-aafc-5e58eccd9d0f  {"api hub": "/ip4/<IP_ADDRESS>/tcp/45132"}

TIP

Using process controll like systemmd or supervisord is recommended.

WARNING

Pls keep the wallet unlock state. If the state is locked , it will block sealer "Waiting for confirmation".

# Install damocles

Download source code.

$ git clone https://github.com/ipfs-force-community/damocles.git

Compile damocles.

$ cd damocles
$ git checkout <LATEST_RELEASE>
$ make all

TIP

After compilation, two executable, damocles-worker and damocles-manager, will be generated under ./dist/bin.

# damocles-manager

Initialize work space.

./dist/bin/damocles-manager daemon init

# Create a miner-id (optional)

Skip this step if you already have a miner-id.

$ ./damocles-manager util miner create 
--from=<OWNER_ADDRESS> 
--owner=<OWNER_ADDRESS> 
--worker=<WORKER_ADDRESS>
--sector-size=32GiB

TIP

Use --net param for different network options. Default is mainnet.

TIP

You will see logs similar to the following once successfully created new miner-id.

2022-05-09T07:29:51.325Z  INFO  cmd  internal/util_miner.go:242  miner actor: t01219 (t2z4nnndtt2iuu57lq3ivor7xdvqwivwb5renrh3a)  
{"size": "512MiB", "from": "t3wultr2i47m24ew2p3bskclnb4rtgmuwb6cwjleqbeu4ndqocx67635dibpsq3ugpbv5y7fpjlpxlmkqcgnva", 
"actor": "t01166", "owner": "t3wultr2i47m24ew2p3bskclnb4rtgmuwb6cwjleqbeu4ndqocx67635dibpsq3ugpbv5y7fpjlpxlmkqcgnva", 
"worker": "t3wultr2i47m24ew2p3bskclnb4rtgmuwb6cwjleqbeu4ndqocx67635dibpsq3ugpbv5y7fpjlpxlmkqcgnva"}

# damocles-manager configurations

Tailor ~/.damocles-manager/sector-manager.cfg to your own hardware. Detailed instructions on each configurations could be found here (opens new window).

[Common]
  [Common.API]
  # Gateway service infos, required, string type
  # Fill in according to the actual situation of the service used
  # If a token is supplied in the format of "{token}:{multiaddr}", then the token supplied would be used to construct the RPC client instead of the common token generated by sophon-auth from Token configuration item.
  Gateway = ["/ip4/{api_host}/tcp/{api_port}"]

  # common token for services, required, string type
  # Fill in according to the actual situation of the service used
  Token = "{some token}"

  # Chain service info, optional, string type
  # Fill in according to the actual situation of the service used
  # If a token is supplied in the format of "{token}:{multiaddr}", then the token supplied would be used to construct the RPC client instead of the common token generated by sophon-auth from Token configuration item.
  # If not set, use value of Gateway as default 
  Chain = "/ip4/{api_host}/tcp/{api_port}"

  # Message service info, optional, string type
  # Fill in according to the actual situation of the service used
  # If a token is supplied in the format of "{token}:{multiaddr}", then the token supplied would be used to construct the RPC client instead of the common token generated by sophon-auth from Token configuration item.
  # If not set, use value of Gateway as default 
  Messager = "/ip4/{api_host}/tcp/{api_port}"

  # Market service info, optional, string type
  # Fill in according to the actual situation of the service used
  # If a token is supplied in the format of "{token}:{multiaddr}", then the token supplied would be used to construct the RPC client instead of the common token generated by sophon-auth from Token configuration item.
  # If not set, use value of Gateway as default 
  Market = "/ip4/{api_host}/tcp/{api_port}"
   
[[Common.PieceStores]]
  # *mandatory*
  #Path = "{store_path}"
   
[[Common.PersistStores]]
  # optional
  Name = "{store_name1}"
  # *mandatory* use of absolute path is recommended
  Path = "{store_path1}"
   
[[Common.PersistStores]]
  Name = "{store_name2}"
  Path = "{store_path2}"
   
[[Miners]]
  # *mandatory* `SP` actor id (Remove the "t0" or "f0" at the beginning)
  Actor = 33680
   
  [Miners.Sector]
    # oprional, pledge new sector from InitNumber
    InitNumber = 0
    # oprional, stop pledging new sector at MaxNumber
    MaxNumber = 1000000
    # Enable sealing
    Enabled = true
    # Enable storage deals
    EnableDeals = true
   
  [Miners.Commitment.Pre]
    # address for sending out pre messages
    Sender = "t3vi4amwofexsfpontn5g722psgikzochthhhu3ptvofzrqmgajs67gt5n2ririlc4hj667dvfsn3kmxiwgtya"
    #GasOverEstimation = 1.2
    # FeeCap limit for single pre message
    #MaxFeeCap = "5 nanoFIL"
   
  [Miners.Commitment.Pre.Batch]
    # if bacth pre is enabled?
    Enabled = true
    # Minimum number of messages for one batch
    #Threshold = 16
    # Max wait time before batch pre
    #MaxWait = "1h0m0s"
    # How often it checks the threshold is met
    #CheckInterval = "1m0s"
    #GasOverEstimation = 1.2
    #MaxFeeCap = "5 nanoFIL"
   
  # Same set of config for ProveCommit as for pre above
  [Miners.Commitment.Prove]
    Sender = "t3vi4amwofexsfpontn5g722psgikzochthhhu3ptvofzrqmgajs67gt5n2ririlc4hj667dvfsn3kmxiwgtya"
    #GasOverEstimation = 1.2
    #MaxFeeCap = "5 nanoFIL"
   
  [Miners.Commitment.Prove.Batch]
    Enabled = true
    #Threshold = 16
    #MaxWait = "1h0m0s"
    #CheckInterval = "1m0s"
    #GasOverEstimation = 1.2
    #MaxFeeCap = "5 nanoFIL"
   
  [Miners.PoSt]
    # address for sending out Post messages
    Sender = "t3vi4amwofexsfpontn5g722psgikzochthhhu3ptvofzrqmgajs67gt5n2ririlc4hj667dvfsn3kmxiwgtya"
    Enabled = true
    #StrictCheck = true
    #GasOverEstimation = 1.2
    # FeeCap limit for single post message
    #MaxFeeCap = "5 nanoFIL"
    #Confidence = 10
   
  [Miners.Proof]
    Enabled = true
   
  [Miners.Deal]
    Enabled = true

Start damocles-manager.

$ ./dist/bin/damocles-manager --net=cali daemon run --poster --miner --listen 0.0.0.0:1789

TIP

Again, use --net param for different network options. Default is mainnet.

# damocles-worker

Plan your sealing storage and init directories.

$ ./dist/bin/damocles-worker store sealing-init -l <dir1> <dir2> <dir3> <...>

Attach permanent storage and init directories.

$ ./dist/bin/damocles-worker store file-init -l <dir1>

Plan CPU cores, numa, etc for eahc sealing tasks and tailor your configuration file to it. The following is an example configuration for damocles-worker. More detailed instructions on each configurations could be found here (opens new window)

[worker]
  # optional, instance name of damocles-manager
  # name = "bytest"
  # rpc_server.host = "0.0.0.0"
  # rpc_server.port = 17890

[sector_manager]
  rpc_client.addr = "/ip4/{some_ip}/tcp/1789"
  # rpc_client.headers = { User-Agent = "jsonrpc-core-client" }
  # piece_token = "{auth token}"

[sealing]
  # miner IDs that this damocles-worker can serve; Ex: [22908, 11034, 191107]
  allowed_miners = [33680]
  allowed_sizes = ["32GiB","64GiB"]
  # When set to ture, you will need to configure `piece_token` in `sector_manager`
  # enable_deals = true
  # Max number of retries when errors of temp type encountred during sealing
  # max_retries = 3
  # Time intervals between retires when errors of temp type encountred during sealing
  # seal_interval = "30s"
  # Time interval between idle `sealing_store`s to start pledging
  # recover_interval = "30s"
  # Used to reduce network congestion during sealing for rpc polling non-time sensitive informations
  # rpc_polling_interval = "30s"
  # wether to skip local proof check
  # ignore_proof_check = false

# Plan out your sealing pipeline based on your hardware specs
[[sealing_thread]]
  # Number of sealing_thread corresponds to number of working directories you configured for sealing storage; Absolute path is preferred
  location = "{path to sealing store1}"
  # Each sealing thread then can be further customized 
  # sealing.allowed_miners = [10123, 10124, 10125]
  # sealing.allowed_sizes = ["32GiB", "64GiB"]
  # sealing.enable_deals = true
  # sealing.max_retries = 3
  # sealing.seal_interval = "30s"
  # sealing.recover_interval = "30s"
  # sealing.rpc_polling_interval = "30s"
  # sealing.ignore_proof_check = false

[[sealing_thread]]
  location = "{path to sealing store2}"

[[sealing_thread]]
  location = "{path to sealing store3}"

[[sealing_thread]]
  location = "{path to sealing store4}"

[[sealing_thread]]
  location = "{path to sealing store5}"

[[sealing_thread]]
  location = "{path to sealing store6}"

[[sealing_thread]]
  location = "{path to sealing store7}"

[[sealing_thread]]
  location = "{path to sealing store8}"

# You can have multiple [[attached]] path for permanent storage of sealed sectors
[[attached]]
  # optional
  name = "bytest"
  # Absolute path is preferred
  location = "/xx/xx/xxx"
  #readonly = false

[processors.static_tree_d]
  32GiB = "{path to static tree_d for 32GiB}"
  64GiB = "{path to static tree_d for 64GiB}"

[processors.limit]
  # Number of concurrent pc1
  pc1 = 4
  # Number of concurrent pc2
  pc2 = 2
  # Number of concurrent c2
  c2 = 1

# fields for pc1 processors
[[processors.pc1]]
  # optional
  numa_preferred = 0
  # follows cgroup.cpuset format
  cgroup.cpuset = "0-7"
  # max concurrency
  concurrent = 2
  # Any external environment variable when executing this particular pc1 task
  envs = { FIL_PROOFS_USE_MULTICORE_SDR = "1" }

[[processors.pc1]]
  numa_preferred = 1
  cgroup.cpuset = "12-19"
  concurrent = 2
  envs = { FIL_PROOFS_USE_MULTICORE_SDR = "1" }

[[processors.pc2]]
  cgroup.cpuset = "8-11,24-27"
  envs = { FIL_PROOFS_USE_GPU_COLUMN_BUILDER = "1", FIL_PROOFS_USE_GPU_TREE_BUILDER = "1", CUDA_VISIBLE_DEVICES = "0" }

[[processors.pc2]]
  cgroup.cpuset = "20-23,36-39"
  envs = { FIL_PROOFS_USE_GPU_COLUMN_BUILDER = "1", FIL_PROOFS_USE_GPU_TREE_BUILDER = "1", CUDA_VISIBLE_DEVICES = "1" }

[[processors.c2]]
  cgroup.cpuset = "28-35"
  envs = { CUDA_VISIBLE_DEVICES = "2,3" }

[[processors.tree_d]]
  cgroup.cpuset = "40-45"

TIP

You can also reference the configurations in damocles community test (opens new window).

Start damocles-worker.

$ /path/to/damocles-worker daemon -c /path/to/damocles-worker.toml

TIP

The above sector-manager.cfg and damocles-worker.toml are just minimal to get you to start pledging sectors. More detailed information on configurations could be found here.

TIP

Note that name = "xxx" under [[attached]] in damocles-worker.toml must be the same Name = "xxx" under [[Common.PersistStores]] in sector-manager.cfg.

# Questions?

Find us on Slack (opens new window)!

← venus daemon →