I decided to skip last week’s progress report, since I wanted to keep the focus on our roadmap post. In the meantime, we’ve continued to make good progress over past two weeks. Below is a summary of the work done over this period:

Hived work (blockchain node software)

As mentioned previously, we’re currently creating a hived plugin that can directly write the needed data into hivemind’s database during hive reindexing and normal block reception. Most of the data being provided by get_block_api is of no interest to hivemind, so using this API to get the data is unnecessarily wasting cpu, in addition to slowing down hivemind. I expect that using the plugin approach will lead to significant speedup in the initial sync time for hivemind (my guess now is 2x at least) and it should also reduce normal hivemind live-sync write time. Work is ongoing here:
https://gitlab.syncad.com/hive/hive/-/commits/km_live_postgres_dump/

We made a change to hived so that proposals aren’t automatically removed after they expire:
https://gitlab.syncad.com/hive/hive/-/merge_requests/154
This allows the API to report on proposals after they have ended.

We used memory-mapped IO to speed up rebuild of the block_log.index file from the block log. It’s now over 3x faster on a fast nvme drive (and the speedup is even better on a magnetic drive).
https://gitlab.syncad.com/hive/hive/-/merge_requests/156

Hivemind (2nd layer microservice for social media applications)

Most of our Hive devs continued to working on hivemind last week. Below are some of the merge requests incorporated into the develop branch of the hivemind repo:

enable decentralized muting:
https://gitlab.syncad.com/hive/hivemind/-/merge_requests/407
https://gitlab.syncad.com/hive/hivemind/-/merge_requests/421
https://gitlab.syncad.com/hive/hivemind/-/merge_requests/422

new tests:
https://gitlab.syncad.com/hive/hivemind/-/merge_requests/415

test system improvement:
https://gitlab.syncad.com/hive/hivemind/-/merge_requests/380

We also ran a full sync test on hivemind (4 day long test currently), which allowed us to catch and fix a few more bugs.

We are currently during real-world testing the head of develop branch on api.hive.blog.

Support for decentralized lists/muting in condenser (https://hive.blog)

We made several fixes and improvements to the UI based on feedback from beta-testers:
https://gitlab.syncad.com/hive/condenser/-/merge_requests/165
https://gitlab.syncad.com/hive/condenser/-/merge_requests/166

What’s the plan for next week?

Finish hivemind and condenser decentralized list changes (likely done tomorrow)
Continue creating hivemind tests (long term task).
Automate running of hived/hivemind full sync tests and run it once a week.
Continue work on speedup of hivemind full sync via hived plugin as the slow sync time has a big impact on the speed of hivemind CI (which sets an upper limit on how fast we can validate changes).
As a stopgap to the above, we’re also trying to speedup the current full sync code by making more concurrent API calls to hived from hive sync (we’ll probably be able to get a 2x speedup this way, in other words, bringing the full sync time down to 2 days on a server with sufficient cores to handle the additional hived load).
Make an official hivemind release to master branch and create a hivemind snapshot so that API node operators can update their nodes quickly. After all API nodes have updated, we can deploy the new changes to condenser for decentralized lists and decentralized muting.

Updated on BlockTrades Hive work (as of Dec 9th 2020)

Hived work (blockchain node software)

Hivemind (2nd layer microservice for social media applications)

Support for decentralized lists/muting in condenser (https://hive.blog)

What’s the plan for next week?