The Steem Bot Tracker website was the first Steem-related development project I've ever done. I built it in a couple days back in early October as an experiment and learning experience using the Steem APIs and to see if I could make something to help me better utilize the voting bots to promote my posts.
At the time I never imagined it would grow to be what it is today, and I also had a lot of learning to do on the best and most efficient ways of utilizing the Steem APIs. Over the past 5 months I added loads of new features on top of a messy and inefficient code base and it was finally time to go back and fix it.
What could have been done better
Previously, the entire site ran in the browser. It was just an HTML page with some JavaScript and there was no server-side component at all. This meant that all of the Steem API calls were made from each client that was viewing the site separately.
When the site first launched not many people used it and there were about 5 or 6 bid-based voting bots in total, so having each client viewing the page make its own API requests wasn't really a big deal. But now approximately 2,500 people use the site each day, with around 30 on at any given time, and there are almost 100 bid-based voting bots (which is kind of ridiculous, but we'll save that for another post).
The site reloads the voting bot info every 30 seconds, so that means that all 30 people with the site open at one time were making API calls to get information for up to 100 voting bots every 30 seconds!
While this is still probably inconsequential relative to the total traffic to the api.steemit.com node, it needed to be fixed. Additionally, when the api.steemit.com node has issues - like it did last Friday - the site wasn't able to load data which caused a number of problems for bot users and owners and there was no easy way for me to fix that other than to wait for the node to be fixed.
The fix
I have been working for the past couple weeks on updating the site to pull data from a central server instead of directly connecting to a full Steem node. This creates a number of advantages. Now the site can get nearly all of the data it needs about all of the bots from one API call to the server, as compared to potentially hundreds of calls before.
Additionally should the api.steemit.com node have issues again the server can easily switch over to another node that is working and the site will be unaffected.
On top of that, the site should now work better for the users. The list of bid bots should load nearly instantly and also update nearly instantly as you adjust the filters since what was previously a large number of Steem API calls has been reduced to one call to get data pre-loaded on a central server.
The code has been generally re-factored and cleaned up as well which should lead to fewer bugs and faster turnaround time for new features as well as making it much easier for others to contribute to the development of the project.
For bot owners
For bid-bot owners, one thing to note is that from now on the site will require that bots implement the bids API in order to display detailed bid information. Previously for bots that did not provide an API the site would try to figure out the bid details from the bot's account history, but that was never very accurate and caused too many problems by displaying incorrect data.
The following are the list of bid-bots which currently do not provide an API:
If the owners of any of those bots need help setting up the API please get in touch with me.
For bot users
This was a mostly behind the scenes change to make the site run better and faster, so for those of you who were hoping for some new cool features - sit tight, they will come soon! These changes were necessary to make it easier to develop new features and keep the site working when the SteemIt node has issues.
That being said, hopefully you notice the site running better and faster now! If you have any new features you would like to see on the site please let me know in the comments.
Thanks for your support!
As always I want to thank everyone who has helped and supported me in creating this tool. I love that it has been able to help so many people learn about and use vote selling services to promote their content and grow their following and earnings on the Steem platform. Once again if you have a suggestion or feature request I encourage you to submit an issue on the GitHub repo for the project. Please stay tuned for more updates in the coming weeks!
For those of you who are interested in looking at the code, there are too many commits to list here, but the updated code is in the "big_changes" branch which you can see here: https://github.com/MattyIce/bottracker/tree/big_changes
Posted on Utopian.io - Rewarding Open Source Contributors