CONTENTS
Witness Server Infrastructure Update
Automated Monitoring Update
Development Projects Preview
Curie operates a community witness on the
account.
100% of witness block production rewards are used for Curie operations.
Any inquiries into the Curie witness should be made in the dedicated #witness-inquiry channel in the Curie Discord community. Join the Curie Discord community here: https://discord.gg/TXrMC4Z
The Curie witness is operated by
, who is the author of this update.
Witness Server Infrastructure Update
We have just migrated our primary and backup witness nodes onto new infrastructure. Those have just finished replaying on the 19.3 stable build and the primary is happily producing blocks. The new primary node is a 128GB DDR3 6 core dedicated server with 1TB of SSD SATA and a guaranteed 1Gbps network connection. Backup is of a similar spec, but with 64GB ram as opposed to the 128 on the primary.
Automated Monitoring Update
One of my main goals as part of becoming the Curie witness operator was to improve monitoring and alerting, not just around the Curie witness specifically, but on witnesses as a whole.
We have set up some basic monitoring that alerts a private Curie Discord group to any issues. In the case of missed blocks we get an alert into a private channel which is monitored by a number of Curie members as well as a Discord direct message to myself, the backup witness operator () and a backup monitor (
). If anything looks problematic, then we have a call out number with which the backup witness operator and the backup monitor can reach me at any time of day or night. We have coordinated schedules so the backup witness operator will take primary responsibility if I am on vacation or otherwise unavailable for any period of time.
This monitoring app, written in nodejs, also produces a daily report for us at 00:00 UTC or can be called via a discord bot at any time to check the status of various systems. So far we have the following info, but we intend to expand on this in future update.
We have also added some log file analysis to this app which checks for certain messages in the witness log files, one of which being latency. If many witnesses are seeing increased latency (currently set to over 3 seconds), we may have a problem, so it's good to notice this and call it out to the rest of the witnesses.
The script also handles missed blocks. If we miss more than 2 blocks in a row, it will automatically update our witness to use the backup.
After reading a recent post by on monitoring and alerting with netdata I decided to try installing this tool and giving it a go. I was very pleasantly surprised by it. We now have many stats on the metrics of our servers, plus I have set up alerting around some of the important metrics
An alerting example is something like this, where a replay was underway and disk space was filling up. Of course, the alerts reset when the replay is completed
These flow into a private discord channel and are monitored as described above. I am just starting to set these up, but we have disk space alerts up and running as well as a load of other stuff that works out of the box with netdata. It's a very cool tool and well worth getting set up on your own servers.
Development Projects Preview
Curie is currently working on two development projects which will be posted about in more detail in the near future in separate development updates on the blog:
SteemLookUp is a filtering search tool of specific interest to curators and general interest to anyone interested in discovering content on the Steem blockchain.
This open source project can be accessed at https://steemlookup.com, and the Github repository can be found here: https://github.com/steemlookup/steemlookup
While this project will be updated on the blog, you can also follow the
blog for more frequent updating. See this post on the
blog for current information on functionality and use: https://steemit.com/steemit/@steemlookup/introducing-steemlookup-filtering-tool
Curie is developing a full-fledged platform for curation groups / guilds / communities.
This platform will include a post submission system and a back-end accessible to members to allow review of posts, with reporting on submissions available to members. This will replace reliance on the Streemian platform for Curie community operations. This will also be open source and open to any interested curation groups / guilds / communities to use. SteemLookUp will be integrated into this platform to allow submission of posts directly from the filtering search tool.