Transaction Generation on the STEEM TESTNET

We had a TESTNET to test HF20, but it turns out that we need much more effective testing. To understand how we can do this better next time, we need to understand what didn't work well this time.

This document will help us to identify what was not covered by the TESTNET and then make it better.

1. Generic background on blockchain & TESTNET

"A blockchain is a Distributed OpenLedger of records, contracts & Transactions between multiple partieis."

An ideal TESTNET must sync Records, Contracts & Transactions in real time thus replicating the state of the MAINNET.

Further this can be summarized as state transfer from one distributed state machine to another resulting in invocation of various automatic transactions and virtual operations. In a real world scenario, this will not be possible and we will have device elaborate mechanisms to mimic real world interactions.

1.1 discussion

List down the transaction types that can be invoked with a tool similar to Tinman. For these cases, including a periodic sync of state from the MAINNET to the TESTNET can be done with Tinman. Further mimicing the user interaction, reaching the TPS that we can expect on the MAINNET etc are going to be the challenges. If we are syncing from the MAINNET it will not be possible sync the current Resource Credits (or earlier bandwidth data). We will be able to get the posts. For the interactions on the posts we will need to device a mechanism and invite the community.

1.2 TESTNET

A TESTNET is simple terms like a "spell checker" which will make sure that we wrote down is grammatically correct. Every software undergoes through rigorous testing before this goes like. Say Twitter and Facebook are too generic examples.

In the blockchain world the testing is done with a version of the blockchain called TESTNET which will have all the features and rules that define the new version of the blockchain. Finding bugs in the TESTNET and postponing release dates is not a bad thing and this is even done for Bitcoin with their Segwit2x

RCA : why Official TESTNET was not enough ?

We have an official TESTNET live from |date| but as it turns out that the TESTNET was not enough. We need to do a root cause analysis (RCA) to understand why the TESTNET was not enough. This RCA will give us a problem statement. Then, we can create a solution which will address the problem.

what we know of the official TESTNET

it was live from |2018-08-25| Hardfork 20 Testnet Details
Extensive automated testing setup using code is present
The Resource Credit (RC) implementation was difficult to test because of unpredictable variables and difficult to reproduce conditions (which are such conditions ?)
volume of transactions on the MAINNET was higher than TESTNET (how much higher ?)
We need to have similar or equal number of transactions on the TESTNET and MAINNET
intelligently filtering and mirroring non-consensus plugins like RC/bandwidth is difficult or not possible

Questions

Date on which TESTNET was live ? ( 2018-08-25 as per the Hive account@steemitdev post)
what is the version of the offiical TESTNET ?
How many transactions are happening in the MAINNET per week ? (7 x 86400 seconds)
How many transactions happened in the official TESTNET ?
What was the highest TPS on the ofifcial TESTNET ?
Can we create scenarios like upvote which will test the official testnet ?
Do we have a list of scenarios and transactions that cannot be effectively tested on the TESTNET ?
so can we identify the difficult areas like RC and test them ?
Which are the difficult scenarios that was not tested by the automated test environment ?