honesty is a good quality for a witness to have and here we have to be hones...
we set up our server, started running the node and everything looked all good under the hood.
But we had forgotten to check one crucial thing. We did not accurately check our latency...
Our setup came from the build that we copied from 
Now we did see that there was high latency and we assumed it came from the other end. We will admit we were a bit confused.
And then we went to bed and woke up with the dreaded missed block on our counter...
We had lost the golden 0
So since we are a still high up the list we will only roughly hit a block per day (maybe two days)
We did something else, we deleted everything and started from scratch.
And now we must have done it right since we went to bed (again) and when we woke up we had the ugly red stripe through our name gone and had processed our first block as witness on the Steem Blockchain.
So here is a small analysis from 
Why did we miss a block?
We had errors in docker logs e.g:
[33m3358102ms th_a fork_database.cpp:44 push_block ] Head: 22194265, 0152a8593f19feb7693cff5c9e5a13776cc187e5
[0m
[31m3358124ms th_a application.cpp:524 handle_block ] Error when pushing block:
4080000 unlinkable_block_exception: unlinkable block
block does not link to known chain
{}
th_a fork_database.cpp:64 _push_block
{}
th_a database.cpp:625 _push_block
Logs showed high latency:
2025705ms th_a application.cpp:514 handle_block ] Got 42 transactions on block 22344986 by curie -- latency: 9705 ms
2029161ms th_a application.cpp:514 handle_block ] Got 63 transactions on block 22344987 by xeldal -- latency: 10161 ms
2031774ms th_a application.cpp:514 handle_block ] Got 45 transactions on block 22344988 by ausbitbank -- latency: 9774 ms
2034703ms th_a application.cpp:514 handle_block ] Got 42 transactions on block 22344989 by aggroed -- latency: 9703 ms
2037698ms th_a application.cpp:514 handle_block ] Got 44 transactions on block 22344990 by gtg -- latency: 9698 ms
2040719ms th_a application.cpp:514 handle_block ] Got 39 transactions on block 22344991 by good-karma -- latency: 9719 ms
2043699ms th_a application.cpp:514 handle_block ] Got 32 transactions on block 22344992 by followbtcnews -- latency: 9699 ms
2046720ms th_a application.cpp:514 handle_block ] Got 47 transactions on block 22344993 by curie -- latency: 9720 ms
2049698ms th_a application.cpp:514 handle_block ] Got 37 transactions on block 22344994 by lukestokes.mhth -- latency: 9698 ms
2052689ms th_a application.cpp:514 handle_block ] Got 43 transactions on block 22344995 by blocktrades -- latency: 9689 ms
2055787ms th_a application.cpp:514 handle_block ] Got 59 transactions on block 22344996 by timcliff -- latency: 9787 ms
2058739ms th_a application.cpp:514 handle_block ] Got 50 transactions on block 22344997 by someguy123 -- latency: 9739 ms
While speedtest against distant server was within reasonable limits
$ speedtest-cli --server 5026
Hosted by AT&T (San Francisco, CA) [9373.31 km]: 444.506 ms
Testing download speed........................................
Download: 33.36 Mbit/s
Testing upload speed..................................................
Upload: 17.37 Mbit/s
When installing the replay command took 10hrs until we have seen “handle_block” in logs
How did we fix this?
Honestly I don’t know, while the logs showed high latency I started steem node from scratch without docker as per guides:
Source : https://steemit.com/witness-category/@krnel/complete-witness-installation-guide
Source: https://steemkr.com/witness-category/@timcliff/steps-for-setting-up-a-witness-node-the-manual-way
Before the manual building I ran commands for required/recommended libraries.
Some of those were already installed (as expected) but there were a few which we haven’t had before although to be fair they were not listed in guide made by
$ sudo apt-get update
$ sudo apt-get install -y autoconf automake cmake g++ git libssl-dev libtool make pkg-config
$ sudo apt-get install -y libboost-chrono-dev libboost-context-dev libboost-coroutine-dev libboost-date-time-dev libboost-filesystem-dev libboost-iostreams-dev libboost-locale-dev libboost-program-options-dev libboost-serialization-dev libboost-signals-dev libboost-system-dev libboost-test-dev libboost-thread-dev
$ sudo apt-get install -y doxygen libncurses5-dev libreadline-dev perl
$ sudo apt-get install git make automake cmake g++ libssl-dev autoconf libtool
$ sudo apt-get install libboost-thread-dev libboost-date-time-dev libboost-system-dev libboost-filesystem-dev libboost-program-options-dev libboost-signals-dev libboost-serialization-dev libboost-chrono-dev libboost-test-dev libboost-context-dev libboost-locale-dev libboost-coroutine-dev libboost-iostreams-dev
$ sudo apt-get install doxygen perl libreadline-dev libncurses5-dev
$ sudo apt-get install ntp
$ sudo apt-get install python3-pip
$ sudo pip3 install jinja2
Then:
git clone https://github.com/steemit/steem
cd steem
git checkout v0.19.2
git submodule update --init --recursive
cmake -DLOW_MEMORY_NODE=ON -DCMAKE_BUILD_TYPE=Release .
make steemd
make cli_wallet
sudo make install
whilst waiting for the installation I have checked the logs again just to see if we have errors and noticed that messages were in green and latency much lower (mostly under 200 except from two which were below 700ms).
20hrs later we successfully processed our first block
So if you would want to help us further in our quest to get more blocks and secure the blockchain then please click on the link below to vote for our witness
https://steemconnect.com/sign/account-witness-vote?witness=swisswitness&approve=1
