Woes of the witness and the salvation the elders bring
Pricefeed
Yesterday my witness server had an issue. The pricefeed failed irreparably.
I have been running he steemfeed-js since my witness went online a bit over three weeks ago. I found that over the last week I was having to restart it over and over. Eventually It got to a point I could not get it to run. The Feed name was locked in the Network Bridge and I could not get it to flush. Error after error, I was done.
Here is where I need to thank 
Memory
Yes, I had to upgrade my configuration today as my witness server had an issue this morning. It ran out of memory.
My witness server is 1 half of a twin 1U Supermicro server, the other half is an in production MySQL server for a client of my hosting company. So I could not shut it down and pull it from the rack to do an upgrade of the the physical RAM. The board was stuck with 32GB Ram. Well I had an ace up my sleeve! A fresh deck of cards was dealt to me by 
My Test Lab
I had a couple of these twin servers laying around from my hosting company and so I started working on my SEED server at home last week on one of them.
I decided to experiment with ZRAM since it was not yet a production SEED server so I could screw up and not be kicking myself for failing in a production environment. I read a post a month ago about making your STEEM server last longer with memory compression. This was my Ace.
Well I built my SEED server and before I ran the replay I installed ZRAM and configured it, I thought. Things went okay for a bit but I kept having an issue at about 19% of the replay. It would just stop running. I dealt with this for a couple of hours and finally posted to the witness channel on Steemit.chat. This is when 
So I took what I learned on my SEED server and applied the ZRAM install to my Production witness server. But I had a problem. My server was up for a block in about 30 minutes and I could not do the install and replay before my block. I needed to disable my witness.
Right now I have not missed a block. My server is running very well and my Network is SOLID. I did not want to screw that little statistic up. I used the suggestion from @Drakos to disable the signing key with “STM1111111111111111111111111111111114T1Anm”. It Worked! Then I got started on my ZRAM upgrade.
I used everything I learned on my SEED server to configure my production witness server. A couple of misteps but I finally got the configuration right and my replay was up and running. 3 hours later it finished and I opened my witness wallet and put the active signing key back in and ran the witness_update and I was back. I may have missed my place in line but thanks to the sheparding from these other gratious witnesses I was able to get back in line and get my next block. I am still at ZERO missed blocks and my Witness server now shows a useable 64GB of ram and ZERO blocks missed.
Tomorrow I plan to clone my SEED server in Proxmox and bring the clone up as my backup witness. I have 4 new 3.0GHz XEON CPU's I am installing in place of the 2.0 running on both sides now.
That will allow me to have a SEED and my backup witness running on a single server using virtualzation and 32 GB physical ram. Now that's hot!
My Current Witness Server Settings
Specs:
8 Core (16 Thread) Intel CPU
32GB RAM with 32 in ZRAM
860GB SSD Storage
Unlimited Bandwidth
Ubuntu 16.04
APR: 0%
Account Creation Fee: 0.1
Block Processing Daily Average: 3.9
Blocks Missed: 0
Price Feed: Updated Hourly using
Current witness rank: 82
Going forward using the knowledge I have now and the access to the rackspace and bandwidth I have, I plan on providing a full 512GB Ram RPC node for public development. I need to see crypto do another parabolic run up to the moon to swing the $9,000.00 for the equipment that the node is going to cost. That is my next commitment to STEEM and the devolpers on the network.
I have some others I would like to thank, (not mentioned above) that helped me along the way. Each of them has earned a witness vote from me because of there assistance.
- @Yehey for providing very well documented easy to follow guides;
Build your witness
Build your seed that gave me an alternative way to access my cli_wallet
that helped me with my original pricefeed configuration using steemfeed.js
for providing the gtihub docker used in
's guides. That Docker script is the BOMB!
- @Jerrybanfield for providing his
program that helps underpaid witnesses below an active rank of 80 with additional steem helping us afford the cost of being a witnesses. I would also like to thank Jerry for all his how to and his "inspirational" videos. I would not even be on STEEMit.com had it not been for @Jerrybanfield. A true inspiration and the one witness I consider the STEEM Evangilist. We all owe Jerry a debt of gratitude for his undying preaching of the STEEM.
Nothing works with out community assistance here. I just want to remind everyone. Pay it forward. If you learn something new post it, share it. Push it up to github if you can.
Just my two STEEMS Worth.
Vote LibertyRanger as your Witness or Set Proxy.
Go to https://steemit.com/~witnesses URL address then scroll down to Vote.
Or you may simply follow this link and click the Continue button to confirm your vote.
https://v2.steemconnect.com/sign/account-witness-vote?witness=libertyranger&approve=1
Please follow me @LibertyRanger, Vote Up and Resteem
Thank you,
@LibertyRanger