For the past half year I have been running and documenting a gridcoin mining experiment. Gridcoin crunchers and readers following along have an understanding of what it is all about and of motivation for this experiment is. Over time things have become so obvious that I don't want to repeat them in every update. Every now and then I get some questions for details on the gridcoin slack channel or from utopian moderators. Most of the times I refer to my blog in general and sometimes certain updates in particular.
History
Gridcoin mining is quite complex because it depends on a lot of variables. Last summer a couple of gridcoiners and myself had the idea that maybe it could be possible to predict the future mining results if certain variables were ignored. Put simply the idea was to predict future returns for one host based on earlier results of that host. Of course preferably based on as little history as possible. Based on the general understanding of the logic behind it all and in theory very little results would be necessary.
I started with a couple of basic article to explain the details, the math behind, and to make sure my own understanding and formulas were matching reality.
The article quickly became a reference for explanation and even managed to serve as a source for the updated whitepaper. From then on the experiment really started and I tried to gather as much data as we guessed necessary.
No data available
BOINC projects have quite some freedom to change how they present and calculate data but most keep defaults as far as RAC goes. The host page for this experiment's host can be found here. As you can see only the current RAC is available from the host page. There is no way to see the history of it. The tasks themselves have some history but that is also limited somehow. From the task overview you can navigate to next 20, and do that a couple of times but then the history also stops.
At the time we assumed little data was needed to make an accurate prediction. To prove that the prediction was accurate we needed data to verify it with. Unfortunately no historical data is available and the little found was of no use because there were no details about how it was gathered.
It became obvious I had to do it myself and in a very controlled manner. I set up a host to run 24/7 with no other significant resource loads. It's backed up by a UPS. We made some considerations about with project to choose in a hope to have a stable environment for as long as possible. Up to now it has had only one interruption in 6 months and it has continually been crunching the same task.
Host details
The data is gathered on a virtualized Microsoft Windows Server 2012 R2 Standard x64 Edition running on a Intel(R) Xeon(R) CPU E5-2630 0 @ 2.30GHz [Family 6 Model 45 Stepping 7] core with 4GB of RAM. The computing preferences are set to use at most 50% of the cores at 100% of the time. The project being crunched is VGTU and this has been the same task since the very beginning: VGTU@Home application for grillage optimization v1.04.
Early discoveries
There is a time dependency in the RAC math that cause RAC to change slowly over time so it takes days and weeks to see changes. After about a month it started to become obvious that practice was not as simple as theory and that prediction was not as accurate as expected. Measured RAC and predicted RAC started deviating. Just have a look at the current chart below how wrong the predictions were. The yellow and blue lines were the estimates made somewhere at the beginning of September. The orange line is the real value.
It seemed more was at hand as outlined in this first article on the matter. The scope of the experiment shifted. Obviously prediction was not possible without understanding the current data. But the current data did not provide enough details to clearly understand it. I proposed the idea of host normalization but we had no idea on how to prove that. From then on the experiment continued not with prediction in mind but to gather as much data as possible to gain a better understanding. Over time new series were added in an effort to gain better understanding.
How to read the charts
- The light blue dots (arrow 1 and 2) are actual RAC samples. Those values are copied from the website and are the values that are used to determine the amount of GRC mined. Remember that there is no RAC history on the project server, only the current value is available.
- The orange line is the calculated RAC. These values are based on the credits rewarded of which a limited history is available on the project server. Because of this limited history it is possible to get a clearer view of the real RAC evolution without actually having to be able to see every change on the project website.
- This orange line needs to be exact as possible so the deviation (error) is plotted as well in the form of grey dots.
- Estimate 1 and estimate 2 are two predictions made based on basic assumptions back in September. It's quite obvious RAC is more than the simple formula.
- The dark blue dots are the points awarded per day. Because tasks take multiple hours the amount of tasks completed varies severely from day to day.
- To better visualize the points per day the dark blue dotted line was added which is a 2 day moving average.
- This seemed not enough so another technique was used. The green dots are the points per second. This has less variation and more smoothness.
- The small green dots are a 4 sample moving average of the points per second
- In an effort to explain the host normalization team RAC was also added as red dots.
Later a second chart was added with extra parameters. The series descriptions are quite self explaining.
And more recently the last chart was added. Here I keep track of the RAC of the top 10 Gridcoin contributors to VGTU. If an account falls out of the top 10 I stop collecting it's data. If it later reenters this causes a gap. As extra parameters I add the sum of these 10 as well as the team RAC.
Updates
I only post updates about this experiment under certain circumstances. I don't have any fixed rules but generally speaking these are the triggers:
- Some new phenomena is noticed
- A trend or trend reversal is showing
- Nothing has happened for a while
How the data is gathered
Every couple of days I go through a process of copying and formatting the data from the project website to Excel. I explained this process in great detail in my last post.