Hi everyone!

I have been secretly working on improving SteemWatch, the notification center for Steem. I gave it as much time as I could, provided that I have a regular job and a girlfriend I need (and want) to attend to :-)

For Users

There are no new features deployed, but the changes on the back-end that I am working on will lead to some of the long awaited features to arrive, eventually. Particularly I am making the whole block processing subsystem modular and more scalable and I want to split it away from the main SteemWatch application as deployed today. This should for example enable me to split content published events into content created and content updated. It's not that it would be impossible to implement now, but I would like to introduce a separate service for every non-trivial piece of logic.

I also started using Telegram, so it is possible that it will push myself to write a Telegram bot and notifier sooner that expected :-)

For Developers AKA Technical Details

I started rewriting SteemWatch from scratch using a brand new repository. Although I consider this monorepo internal to SteemWatch in the way that I am not going to try to keep the package APIs stable at all cost, but there are definitely certain packages and components that could be interesting for people outside.

I should note that everything that follows is relevant for steemwatch@13f645db8fda69f588fa617bd2fe10390d1f0464.

Block Processor

The executable stored in cmd/block_processor processes block operations as stored on the blockchain and forwards them to a NATS Streaming channel, encoded using Protocol Buffers. It uses at-least-once delivery semantics, i.e. every block is processed at least once.

block_processor also uses get_ops_in_block instead of get_block, so SteemWatch is ready to potentially start processing virtual operations as well and turn them into events that are emitted.

Anybody can run this component as a standalone service in their own infractucture.

Protocol Buffers for Steem

This leads me to the package that can be found in pkg/pb/steempb. It contains Protocol Buffers definitions for blockchain operations and such along with the code generated for Go.

Again, anybody can re-use these definitions and get Protocol Buffers definitions for free. It was pretty annoying to write it all, so spare yourself the suffering and use it!

Mock steemd RPC Endpoint for Testing

When implementing block_processor, I decided that I am going to do things properly, i.e. write some test. So I implemented a simple end-to-end test. While doing so, I ended up implementing a simple testing package for mocking the steemd RPC endpoint. The package can be found in pkg/testing/steem, and this is how it looks like when you want to mock some database_api RPC method handlers:

// This is github.com/go-steem/rpc/apis/database.
db := database.NumbericAPIID

server.HandleRPC(db, "get_config",
    func(apiID int, method string, args []*json.RawMessage) (interface{}, error) {
        return map[string]interface{}{
            "STEEMIT_BLOCK_INTERVAL": 1,
        }, nil
    })

server.HandleRPC(db, "get_dynamic_global_properties",
    func(apiID int, method string, args []*json.RawMessage) (interface{}, error) {
        return map[string]interface{}{
            "last_irreversible_block_num": 1,
        }, nil
    })

The mock service being used for block_processor, which is a complete example, can be found in cmd/block_processor/tests/steemd.

Summary

I am personally pretty excited to start playing with a better architecture and interesting technologies. The changes introduced open new possibilities for the future, even though they are pretty invisible to the regular users.

What is Next?

The next step is to implement the service that consumes messages as generated by block_processor and turns them into events as you know from SteemWatch. I am planning to implement this service so that it keeps the whole user settings context in memory. The current implementation consults the database for every event being emitted to check whether there are any users interested, which is pretty slow and inefficient, particularly because it is impossible to create the necessary indexes in MongoDB. I would like to keep it in memory and reload only when users modify their settings through the web app.

Support

No need to send money, spread the word about SteemWatch (e.g. resteem) and use it yourself!