Setting up a Witness - HF20 September 2018

Setting up a Steem Witness server? Here is an updated guide for September 2018, just after Hard Fork 20 took effect. I wrote this guide as I was setting up my own server in the same time, because there weren't really any guides that were entirely up-to-date and also had the full information available there. At the bottom, I'll list the guides that I used to put this one together. I also decided to manually set it all up, instead of using the docker image, which certainly contributed to my struggles but should not have contributed so much overall.

I use vim as my text-editor throughout this guide, but use whatever you're comfortable with.

Are you ready to become a witness?

Make sure you understand all of the requirements of witnesses, as well as why witnesses are needed and what they do other than running their servers. Check out Hive account@thecryptofiend's witness questionnaire and see if you can answer all of the questions. Talk to some of the other witnesses on steemit.chat in the #witness channel and read about what other witnesses do for the community. Once you're sure, continue on with the guide.

Requirements

To become a witness, you're going to need several things. Most importantly, you're going to need at least two servers, one for a witness node and one for a seed node, and you need the skills on the command line and with servers to keep them up and running. You also need a Steem account (obviously), some time to set it all up, and time in general to support the community and maintain your server.

Step 1: Hardware

To run a witness and seed node, you're gonna need servers (obviously). Right now the block logs around 147GB, and the shared memory file is around 53GB. These numbers are going to increase over time, so don't go off these if you're reading this in 2019+. I'm running two different servers for my nodes. The witness node is running on an OVH server, with the following specs:

• CPU: Intel Xeon D-1520 - 4c/8t - 2.2GHz /2.7GHz
• RAM: 64GB DDR4 ECC 2133 MHz
• Disks: 2x480GB SSD

My seed node is currently spinning up the blockchain on the following specs:

• CPU: Intel Core i7-7700HQ
• RAM: 32GB DDR4
• Disks: 1x256GB SSD, 1x1TB HDD

The seed node is technically running on my gaming laptop, but I don't do a lot of gaming so that's where it's gonna be until I need to upgrade or have the money to rent a second server.

Step 2: OS & Server Hardening

First, you need to install Ubuntu. I'm assuming you're going with 16.04 LTS, since it's the best supported. You need to have a certain level of skill to run a witness node, so I'm going to assume you know how to install Ubuntu as well. If you don't have a lot of RAM, make sure to make your swap space big enough to make up for it.

Hardening Your Server

You want to make sure that your server is secure, both to be trustworthy and so you don't lose out on your rewards. Here's a basic list of things to do:

1. SSH into the server as your initial login.
2. Create a new user account if the only available user is root: sudo adduser steem
3. Add the new user to the sudoers group: sudo usermod -aG steem
4. Disable your root login and change the port for SSH: sudo vi /etc/ssh/sshd_config

• Find the line #PermitRootLogin no and remove the # to uncomment it.
• Change the line Port 22 to any port available, usually 1000-8000. Make sure you remember the port, because you have to use it as a -p option to SSH in now.

1. Remove IPv6 listening on your server:
   echo 'AddressFamily inet' | sudo tee -a /etc/ssh/sshd_config
2. Restart SSH: sudo service ssh restart
3. Install a basic firewall and only allow the SSH port you chose:

• sudo apt-get install ufw
• sudo ufw default allow outgoing
• sudo ufw default deny incoming
• sudo ufw allow ssh
• sudo ufw allow # make sure to allow the port you changed in sshd_config
• sudo ufw enable

Now generate and add an SSH key so they can't brute-force your password:

1. On your local machine, generate a new key (or use an old one):
   ssh-keygen -t rsa -b 4096.
   Generally you want to use a passphrase with your key as well.
2. Copy the key over to your server:
   ssh-copy-id steem@YOUR_SERVER_IP
3. Test that you can login using the key using SSH, because we're going to disable the password logins.
4. SSH in again, and open the sshd_config file:
   *sudo vi /etc/ssh/sshd_config

• Find the line #PasswordAuthentication yes, change it to no and uncomment it.

1. Restart the SSH daemon:
   sudo service ssh restart

That should be it for hardening the server. Overall, that makes it pretty hard for it to be compromised. Make sure you backup your SSH key, and remember the passphrase for it, and the port for SSH.

Step 3: Installing Dependencies and Making Steemd

Of course, you're going to need a lot of software to run steemd because it's a massive codebase. You need to install all of the dependencies before you try to build it, this is the one-liner I use, some aren't required but useful:

sudo apt-get install -y git make automake cmake g++ libssl-dev autoconf libtool libbz2-dev libsnappy-dev pkg-config libboost-thread-dev libboost-date-time-dev libboost-system-dev libboost-filesystem-dev libboost-program-options-dev libboost-signals-dev libboost-serialization-dev libboost-chrono-dev libboost-test-dev libboost-context-dev libboost-locale-dev libboost-coroutine-dev libboost-iostreams-dev doxygen perl libreadline-dev libncurses5-dev ntp python3-pip screen htop

Now you need the source code for Steem! Just clone the git repository:

1. git clone https://github.com/steemit/steem
2. Generally you want the stable version, but right now the master is actually the right one so make sure you've got it: cd steem; git checkout master
3. Set the options to make the binaries:
   cmake -DLOW_MEMORY_NODE=ON -DCMAKE_BUILD_TYPE=Release -D_CLEAR_VOTES=ON
4. Build steemd and cli_wallet:

• make steemd
• make cli_wallet
• sudo make install

1. Do a quick steemd -v to make sure you have the correct version running.

That's it! You've got them built and the binaries are available to use. Now you just have to set it up and starting syncing the blockchain.

Step 4: Setting Up Your Witness

In my opinion it's better to get all of your things set up before you start running the server, because steemd takes up processing power obviously, and it's easier to troubleshoot some things before you're all the way in.

Setting Up Your Wallet

Set up your wallet and your witness key first, since you'll need these for later steps. You'll want to be in your home directory for this.

1. Run cli_wallet -s SERVER to connect to a node.

• Currently working servers below:
  • wss://gtg.steem.house:8090
  • wss://steemd.steemitdev.com
• To get a server added send me the address and I'll test and add it if it works.

1. Set your password for your wallet (and write it down!)
   set_password 'your_password'
2. Unlock your wallet:
   unlock 'your_password'
3. Go to your steem account and take your active private key, then import it to your wallet:
   import_key ACTIVE_PRIVATE_KEY
4. Generate a new WIF Private Key:
   suggest_brain_key

• Save all of the resulting keys

1. Close the wallet (just ctrl+c)

Configuring Your Node

Now you need to set up your nodes configuration file, config.ini. Run steemd once to get it started and to create the config file, then ctrl+c to stop it. If you let it run normally, your config file will be ~/.steemd/config.ini. If you set up your own data directory, it will be in the top-level. I'm just going to list the things to change here.

• Change plugins = to:
  • plugins = webserver p2p json_rpc witness
  • plugins = database_api witness_api rc_api
  • I have it on two separate lines like that, it works that way not sure if it needs to be.
• shared-file-dir = /dev/shm or wherever your shared files are going to be.
• shared-file-size = 64G make sure this value is correct for your hardware.
• flush-state-interval = 100000
• For the p2p seed nodes, here is a list of the ones that I tested as working:

  p2p-seed-node = anyx.co:2001 seed.jesta.us:2001 seed.roelandp.nl:2001 seed.brandonfrye.us:2001 seed.xeldal.com:12150 seed.steemnodes.com:2001 seed.liondani.com:2016 gtg.steem.house:2001 5.9.18.213:2001 lafonasteem.com:2001 seed.rossco99.com:2001 104.199.118.92:2001 seed.steemviz.com:2001 steem-seed.lukestokes.info:2001 seed.steemian.info:2001 seed.followbtcnews.com:2001 seed.curiesteem.com:2001 seed.riversteem.com:2001 148.251.237.104:2001 seed.jamzed.pl:2001 seed.firepower.ltd:2001

• webserver-http-endpoint = 0.0.0.0:8091
• webserver-ws-endpoint = 0.0.0.0:8090
• witness = 'your_name'
• private-key = WIF_PRIVATE_KEY from suggest_brain_key earlier
• witness-skip-enforce-bandwidth = true
• rc-skip-reject-not-enough-rc = false from hard fork 20's new RC system.

That's basically all of it, you may have to configure it some more if you're doing extra things on top of witnessing on this server.

Syncing the Blockchain

There are two ways to go about getting the blockchain after you've set it all up.

Simple Sync

Just download it from the p2p seed nodes, which takes a bit longer than the other way:

• screen -S steemd-this sets up a seperate screen to run steemd on
• steemd --resync-blockchain

And just let it run from there. After it's synced up, and you start seeing transactions counts for each block, continue on to starting to witness.

Faster Download

Here you download the block_log from a trusted source, then replay the blockchain, which is faster than a resync.

cd .steemd/blockchain

rm -f block_log block_log.index

wget https://gtg.steem.house/get/blockchain/block_log

As of right now, the block log is around 148 gigs, so it took me about 2.3 hours to download on my server. After it's downloaded, you just replay the blockchain.

screen -S steemd
steemd --replay-blockchain

Starting to Witness

Now that you've synced up or replayed the blockchain, you need to enable your witness, and start sharing a price feed. If you're still on the screen with steemd running, using ctrl+a then d to detach. Right now I use a tool called conductor to do both of these things. It's based on steem-python, and not fully updated for HF20 yet but it'll do the job for you.

1. Install some dependencies: sudo apt-get install libffi-dev; pip3 install steem
2. Install Conductor (if it doesn't work on this link, I'll update to my updated version): pip3 install -U git+https://github.com/Netherdrake/conductor
3. Add the key to your steempy wallet: steempy addkey

• It will ask for your active key and a passphrase.
• It's nice to export your passphrase for this: export UNLOCK='your_password'

1. Initialize your witness: conductor init

• It will ask for your name and witness URL, then run through setup for you if you have not witnessed before. Enter your public signing key, not your WIF private key.

1. To ensure it's working, run: conductor enable YOUR_PUBLIC_KEY to enable your witness.
2. Start a price feed in another screen:

• screen -S feed
• conductor feed
• It will start by showing some prices, then will update your price feed if the spread is high enough.
• ctrl+a then d out

If the feed publishing does not work at the beginning due to the ignored spread (old price: 0.000), open cli_wallet with one of the earlier nodes, unlock, and run: publish_feed "your_witness" { "base":"0.XXX SBD", "quote":"1.XXX STEEM"} true and exit.

Finishing Up

You're done getting started now! Go vote for yourself (and me!) as witness, join up on the witness steemchat, start posting witness updates, and keep up with the community!

The following guides were used in conjunction to create this one:

• Hive account@timcliff's guide the manual way
• Hive account@bitcalm's older guide
• Hive account@krnel's complete guide
• Hive account@bitcoinparadise's low-cost guide
• Hive account@brandonfrye's 2018 beginner's guide
• Hive account@steemed's essential guide

If this has helped you and you'd like to see more content like this, vote for me as witness, upvote, and comment about what content you'd like to see!