Last week we continued work on post HF24 optimizations. Below is a summary of the work done last week and our plans for the upcoming week.
Hived work (blockchain node software)
As mentioned last week, we’re currently creating a hived plugin that can directly write the needed data into hivemind’s database during hive reindexing and normal block reception. Most of the data being provided by get_block_api is of no interest to hivemind, so using this API to get the data is unnecessarily wasting cpu, in addition to slowing down hivemind. I expect that using the plugin approach will lead to significant speedup in the initial sync time for hivemind (my guess now is 2x at least) and it should also reduce normal hivemind live-sync write time. Work is ongoing here:
https://gitlab.syncad.com/hive/hive/-/commits/km_live_postgres_dump/
Hivemind (2nd layer microservice for social media applications)
Most of our Hive devs continued to working on hivemind last week. Below are some of the merge requests incorporated into the develop branch of the hivemind repo:
various bug fixes:
https://gitlab.syncad.com/hive/hivemind/-/merge_requests/381
https://gitlab.syncad.com/hive/hivemind/-/merge_requests/382
https://gitlab.syncad.com/hive/hivemind/-/merge_requests/372
https://gitlab.syncad.com/hive/hivemind/-/merge_requests/387
https://gitlab.syncad.com/hive/hivemind/-/merge_requests/401
https://gitlab.syncad.com/hive/hivemind/-/merge_requests/399
https://gitlab.syncad.com/hive/hivemind/-/merge_requests/404
enable decentralized muting:
https://gitlab.syncad.com/hive/hivemind/-/merge_requests/385
https://gitlab.syncad.com/hive/hivemind/-/merge_requests/392
enable reputation calculation:
https://gitlab.syncad.com/hive/hivemind/-/merge_requests/393
new tests:
https://gitlab.syncad.com/hive/hivemind/-/merge_requests/368
https://gitlab.syncad.com/hive/hivemind/-/merge_requests/361
https://gitlab.syncad.com/hive/hivemind/-/merge_requests/388
https://gitlab.syncad.com/hive/hivemind/-/merge_requests/389
test system improvement:
https://gitlab.syncad.com/hive/hivemind/-/merge_requests/386
https://gitlab.syncad.com/hive/hivemind/-/merge_requests/390
https://gitlab.syncad.com/hive/hivemind/-/merge_requests/397
performance optimizations:
https://gitlab.syncad.com/hive/hivemind/-/merge_requests/394
https://gitlab.syncad.com/hive/hivemind/-/merge_requests/395
https://gitlab.syncad.com/hive/hivemind/-/merge_requests/377
Setup an additional system for api server testing
We bought and setup one of the new Ryzen 9 5900x systems to see how it performs as an API node server. It has good single-threaded performance, so it will be interesting to see how it handles hived reindexing and hivemind full sync (neither of these tasks are particularly parallel right now and they take a fair amount of time).
What’s the plan for next week?
- Finish hivemind and condenser decentralized list changes.
- Create a hivemind database dump for other API node operators.
- Continue creating hivemind tests.
- Run hived/hivemind full sync tests on new server.
- Continue work on speedup of hivemind full sync via hived plugin as the slow sync time has a big impact on the speed of hivemind CI (which sets an upper limit on how fast we can validate changes).
- Write a post with plans for HF25.