How are the witnesses performing? Let's look at some statistics!

A couple days ago I wrote a post about various ways in which witnesses could abuse (use?) their positions to extract more profit from the system than they're typically meant to. How would we know if a witness is abusing their power? The short and uninteresting answer is that it depends on how they're abusing it. One of the ways a witness can abuse their position is to manipulate the order of transactions that are included in the Steem database, slipping extra votes in or sneaking a market order in milliseconds ahead of some other big operation. I don't know exactly how much money there is to be made doing something like this, but if it's happening, shouldn't we know?

The power of statistics compels you!

The witnesses produce blocks in basically random order. This means that we should be able to assume that every witness has the same probability of seeing any particular transaction at any particular time. Thus, if we record all of the transactions that are included in blocks by all of the witnesses over a long-enough period of time, all of the statistics of these transactions should be identical from witness to witness. If a witness is inserting extra votes in their blocks, it should show up in the statistics!

That is, we shouldn't be able to tell the witnesses apart on the basis of the transactions they include in blocks unless something funny is going on. So let's do it!

Setup:

I wrote a python script that scans the blockchain and records the total of each operation type included by each witness. I had it scan 1,000,001 blocks (starting at number 17 million and stopping at 18 million), and then I plotted some numbers. I think the results are encouraging! But maybe we should ask some questions about what Hive account@thecryptodrive and Hive account@netuoso are doing.

First, for reference, a plot of the total number of blocks produced by each of the top-25 producing witnesses over that interval. This plot shows exactly what we'd think: the top 20 or so witnesses who have (approximately) permanent seats at the witness table all produce almost exactly the same number of blocks.

Now let's do a quick gross test of performance; we'll check the number of transactions included per block. Here, if a witness differs from someone else, it wouldn't be even remotely obvious why.

What's happening there with Hive account@thecryptodrive? About 10% fewer total transactions than everybody else. That's a pretty big discrepancy, and something that we should probably look at more closely and try to understand why it's happening.

Fortunately, if Hive account@thecryptodrive were front-running or inserting extra transactions, you'd expect their number here to be higher, not lower. So now for some more plots.

In the next few, I just picked some interesting operation types and plotted the distributions of those (normalized by the number of blocks produced by each witness). In all of them, Hive account@thecryptodrive will show lower numbers than the others; this isn't indicative of anything we didn't already know. First, vote operations:

Nothing funny here! I was pleased when I created these plots that they look so much like a nice clean uniform distribution (I think; I haven't done any actual numerical statistical tests -- so it's totally possible that I'm missing something important). Next, market orders:

Here, we have a very large spike by Hive account@netuoso, which at the outset seems fishy. But don't go pulling your witness votes from Hive account@netuoso yet; if you look back at the first plot, you can see that Hive account@netuoso was not at the time a permanent witness. That is, Hive account@netuoso has produced many fewer blocks than the other witnesses, so the high proportion of market orders may be misleading. The same is true of Hive account@liondani and Hive account@wackou with their relatively-low numbers of market orders.

Where to next?

I'm going to re-run the scripts to look at more recent data. If my data shows that Hive account@thecryptodrive and Hive account@netuoso are actually doing something different from the other witnesses, this will show up in newer data as well. And of course, even if they're doing something different from the other witnesses, this doesn't immediately imply malfeasance. We'll need a closer look to determine that.

Cheers!

EDIT: I've determined that the high rate of market transactions by Hive account@netuoso is almost certainly totally benign. I plan to publish more details in a coming article.