An APM + log monitoring is needed for the official TESTNET
An APM which can monitor and report events is needed for the official TESTNET to effectively co-ordinate community testing.
NOTE: originally suggested this, tested and verified the effectiveness. I am just posting it again & creating an issue in github
Background - from the BOFH days
Have you ever had a BOFH ?
From early days of UNIX its common to collect logs via mechanisms like syslog and also use centralized mechanisms which are as simple as syslog-ng. Further log aggregation over FTP and various other means were used in all sorts of environments like Telecom, Networking and obviously others.
Now, things have changed and with Agile Software development with well controlled development practices which can track down the changes, automate many parts of the application life cycle the BOFH is nothing but an extinct species.
Really ?!
Welcome to community driven TESTING of STEEM blockchain
Testing any application also includes looking for the "results". In many cases, the results may not be visible in the front end and you will have to peek inside the logs to get an exact picture. For example, was one such curious kid who was looking everywhere and found some peculiar issues. Though he will have a different version of the truth, one of the magical (yea, from wizard sorts) tool he had was an APM. Though existence of such tools are now part of BOFH lore, he made it a reality in our realm. So I believe I have made it quite evident that we need some magic before requesting for it in the OFFICIAL TESTNET.
Magical deeds performed by is listed in his various posts.
(Little discovery thanks to the use of Sumo Logic on my witness server)[https://steemit.com/witness-category/@quochuy/my-sumo-logic-dashboard-is-now-tracking-few-more-steem-block-production-events]
Brief (what can APM do ?)
- An APM can give a near real time view of the logs
- In many cases an APM can group events based on conditions & give insights in easy to understand format
- As we group similar events, new errors will be reported in different groups making it easy to understand new ones (FCAPS)
- In the TESTNET environment, as we generate more and more events/transactions, special cases can be easily monitored
Why not plain old log aggregation ?
I must admit that I am spoiled by habit and often go for plain old "scp | tail -f" combo but any APM starting from the good old Splunk, Newrelic (with its limited C++ support), Sumologic, Dynatrace any of them will be much more effective in handling the events and also presenting them in a logical way.
Considering APM is quite new (?) in our context , can we atleast start with a mechanism to fetch the logs via syslog from the TESTNET environment and then improvise it ?
Draft Architecture for Log management
The purpose of the log management is to provide a mechanism for community to subscribe to the logs and thus events on the TESTNET in a secure manner.
In the STEP1, a means to subscribe for logs can be achieved.
STEP2 : A suitable APM can be deployed
Series Back links
: Gamify Steemit's future Soft/Hard Forks testing - The SteemCommunity Organization
- Witness update : progress on testing, environment setup
- Community TESTNET update 17.20.2018
- issues raised by
on negative RC, APM using Sumologic also deserves much needed mention.