Last week we continued work on post-HF24 optimizations. Below is a summary of the work done last week and our plans for the upcoming week.
Hived work (blockchain node software)
We’re currently finishing up a major optimization to the get_block_api plugin that should likely provide a big boost in performance for the get_block API call (this will likely enable us to speed up the hivemind “full sync” process as well). The old implementation used an overly pessimistic mutex locking scheme that severely degraded potential performance under loading.
We’ve also added a new API call to fetch a consecutive set of blocks in one disk read operation.
We’ll likely commit those changes late today or tomorrow.
Hived status
We plan to tag v1.24.7 as soon as we complete the optimizations to the get_block_api plugin. V1.24.7 will be a recommended upgrade for API node operators, but it doesn’t contain changes needed by witness nodes or exchanges.
Hivemind (2nd layer microservice for social media applications)
Performance optimizations and code cleanup:
https://gitlab.syncad.com/hive/hivemind/-/merge_requests/339
https://gitlab.syncad.com/hive/hivemind/-/merge_requests/329
https://gitlab.syncad.com/hive/hivemind/-/merge_requests/347
https://gitlab.syncad.com/hive/hivemind/-/merge_requests/357
https://gitlab.syncad.com/hive/hivemind/-/merge_requests/337
https://gitlab.syncad.com/hive/hivemind/-/merge_requests/365
https://gitlab.syncad.com/hive/hivemind/-/merge_requests/366
Significant speedup in unread_notifications code:
https://gitlab.syncad.com/hive/hivemind/-/merge_requests/348
https://gitlab.syncad.com/hive/hivemind/-/merge_requests/353
Various bug fixes:
https://gitlab.syncad.com/hive/hivemind/-/merge_requests/346
https://gitlab.syncad.com/hive/hivemind/-/merge_requests/358
https://gitlab.syncad.com/hive/hivemind/-/merge_requests/359
https://gitlab.syncad.com/hive/hivemind/-/merge_requests/362
https://gitlab.syncad.com/hive/hivemind/-/merge_requests/360
https://gitlab.syncad.com/hive/hivemind/-/merge_requests/354
We added API tests for a hivemind “full sync” test run. Full sync test runs are very time-consuming, so these will only be run periodically (e.g. once a week).
https://gitlab.syncad.com/hive/hivemind/-/merge_requests/293
We also introduced performance benchmarking into the automated build-and-test (CI) system. This will help prevent performance regressions as changes are made to the implementation of API calls and/or changes to hivemind’s database schema.
https://gitlab.syncad.com/hive/hivemind/-/merge_requests/339
https://gitlab.syncad.com/hive/hivemind/-/merge_requests/350
https://gitlab.syncad.com/hive/hivemind/-/merge_requests/352
We’ve also been making further improvements for monitoring the performance of hivemind on our production system using various public domain tools for postgres.
Hivemind status
We deployed most of the above changes to hived and hivemind to our production node, api.hive.blog, and they’re working well.
Condenser + Condenser wallet (open-source code for hive.blog)
There’s a known issue with editing comments. We thought we had fixed this issue, but we’re still seeing it in production. It’s a high priority issue, so we’ll likely fix it tomorrow.
What’s the plan for next week?
- Finish re-merge of muting changes related to decentralized lists (some of this work needs to be re-incorporated due to two large divergences in the hivemind code base as part of the work to move notifications and reputation out of hived). I expect we will complete this in the next couple of days. This is one the largest remaining performance issues for hivemind (use of the old global mute lists can have a very negative impact on performance serving up posts).
- Deploy some pending merge requests and analyze performance.
- Enable reputation calculation code.
- Continue creating hivemind tests.
- I still haven’t had time to write up plans for new post-hardfork features, but I’m hopeful I will have time in the next couple of days to get to that.