This is an optional update for the Steem database backend, called MIRA.
MIRA is a Multi Index Rocksdb Adapter that updates the database backend of Steem from memory mapped files to a disk log-structured merge-tree implemented via the RocksDB library.
Reindexing
0.20.11 requires reindexing if you want to use MIRA for the database backend of Steem.
Overview
MIRA is enabled via the cmake build options, ENABLE_MIRA=ON. This build option replaces the experimental build option ENABLE_STD_ALLOCATOR_SUPPORT. Due to the nature of C++ dynamically allocated data structures and the previous memory mapped database, MIRA is not compatible with the old database at runtime and must be enabled/disabled at compile time.
A reindex will be required to rebuild state in MIRA. The same command line argument, --replay-blockchain is used. Additionally, reindexing can be done entirely in memory with MIRA to improve reindex performance using the command line argument --memory-replay. This will keep all indices in memory until the end of the reindex. This can require a significant amount of memory. In the config file, memory-replay-indices can set which individual indices should be kept in memory to keep memory usage down while still benefitting from the increased performance.
RocksDB has many performance tuning options available. We have exposed a subset of those options in a new config file, database.cfg which lives in the data directory. We have provided a tuning guide to get node operators started on configuring their nodes for their hardware.
RocksDB does open a considerable number of files. You may need to configure your OS to allow for so many files to be open at once. These resources can help walk you through that process. ( OS X, Linux )
Impact
In our testing, memory requirements have decreased from 64G for consensus nodes to 16G when tuned correctly. Additionally, we were able to migrate from attached NVMe drives to network attached SSDs with similar, if not better, performance on a live node. Reindex performance with MIRA has much less variability. What this means is that it is much slower when compared to reindexing with an attached NVMe, but considerably faster than a network attached SSD. Your actual performance gain/loss may vary and are largely dependent on the hardware you were running on.
