Something special happened last night that other blockchains could only dream of...
An edge-case bug that effectively halted the EOS Mainnet was diagnosed, patched and deployed within 6 hours, allowing the global EOS network to continue as normal.
In any other traditional proof-of-work blockchain, such as Bitcoin or Ethereum it would have taken days to co-ordinate all users and mining pools to update their clients with new software.
With EOS’s Delegated Poof of Stake (DPOS) which uses elected Block Producers, we already know who to get in touch with to co-ordinate a response and updates to the core software only have to be performed by the block producers to urgently patch and deploy a fix.
And this is exactly what we saw - the EOS community should be proud of how the BP community reacted:
- The top EOS Block Producers and Standbys maintain a secure communication channel for exactly these types of events and we joined the emergency global response call to see what we needed to do.
- Standby Block Producers such as ourselves we’re asked to disable our Block Producer nodes and save any historical data that may have been useful for diagnosis.
- A live update telegram channel (https://t.me/eosmainnet) was setup to keep the community updated regularly as the incident progressed.
- The block producer community basically diagnosed the issue and made preparations to restart the network once a fix had been formalised.
- block.one developers including Dan Larimar we’re invited to join the call and confirmed the issue together with the Block Producers. block.one helped develop and test a patch that was released and then incorporated in to the community EOS-MainNet software.
- Block producers deployed the patch, confirming that normal block production was progressing and opened up public endpoints to allow the community to sync up.
The time from initial diagnosis of the incident to resolution was under 6 hours!
What we as a community can we take away from this...
- Our software is new and there may be other edge-case bugs, but the inbuilt consensus safety mechanisms of the EOS software that effectively halted the chain worked.
- EOS’s unique DPOS and Block Producer community has proven it’s capability to quickly respond and resolve incidents, orders of magnitude quicker than most other decentralised blockchains.
We’d like to thank all the Active and Standby Block Producers who responded so quickly, along with block.one who together made this event a non-issue and showed the world the power of DPOS and the EOS Community.