Adding my 2c to hived's Optimizations In progress.

First one - pool allocators - in principle does not reduce memory usage (at least not at current stage). In fact it increases it somewhat, however due to less fragmentation actual usable memory increases, hence less allocation is reported. It is the same effect we could already get by loading snapshot, except now the benefit persists instead of degrading over time. I'm pretty curious about the performance increase, especially in combination with just finished undo session optimizations. As far as I know pool allocators are not yet applied to objects created for undo session, so there is still room for improvement (might even show up in performance of regular sync, not just in extreme traffic in live sync and in colony+queen cycle). Also we have detailed data on average amount of objects needed at any given time which might allow some tweaking of pool sizes for each individual index, which might also slightly improve overall performance still. Once the MR is included in develop further task opens up, but Marek wanted to first get the estimation of how much actual memory consumption reduction we can gain with it, especially considering that the biggest item is getting much smaller with second mentioned optimization - comment archive.

As for "comment archive" optimization, drastic reduction of memory usage is obviously expected, but the first benchmarks look too good to be true. Even the extreme pessimistic scenarios are faster than current version, with most of comment accesses to archive in RocksDB instead of fresh comments inside memory multiindex (measured on big blocks with over 14k votes in each block). It is not entirely impossible, because RocksDB might be algorithmically faster even though each access to data is potentially much slower, but that would mean it makes sense to take a deep look into type of multiindex we are using, and not just for comments - more opportunities for optimizations :o)