Introduction
The 21st century witnessed great technologies and one of them is Blockchain Technology. Blockchain technology is based on a peer to peer network, it is an immutable public record of data which consists of blocks, and chained together to form a ledger and this ledger is distributed in nature. Any node in the peer to peer network can become a miner, can mine the block. It is distributed and decentralized and no third party is required in its conduct. The decentralization is the essence of blockchain technology. Being a distributed network, it takes a little longer time to process a transaction as compared to centralized payment system. However, over the last few years’ significant improvement have been made to improve transaction throughput.
A public blockchain has to be secure, decentralized and highly scalable to become an underlying platform for almost all types of entities and business enterprises. The existing blockchain solutions are either “scalable & somewhat centralized” or “decentralized, secured but not scalable”. Bitcoin scales at a rate of 7 tps, ETH at a rate of 20 tps. Some recent blockchain projects scale better compared to BTC or ETH but they are partially centralized.
In blockchain, the tradeoff between the two(scalability & ecentralization), is also creating a dilemma in real world application. There are many economies, industries, institutions who are eagerly looking for a blockchain solution that can promise all the three fundamental attributes(security, decentralization, scalability). The application feasibility of blockchain technology has a lot to do with the scaling issues. This is one area in which Blockchain is far behind the centralized system. If that can be mitigated, then Blockchain can replace the traditional centralized system at a faster rate & a new tradition of large decentralized marketplace can be established and then the mass adoption will be an automatic phenomenon.
Taking motivation from this & with an aim to extend the reach of blockchain technology, to expand the possibility of large decentralized marketplace & to make it a blockchain platform for Billions, Harmony offers a blockchain solution with state sharding technology, with a dynamic sharding structure, effective PoS, etc to make the network provably secure, scalable, cost effective and decentralized.
Harmony
Harmony leverages on "state sharding" to make the blockchain decentralized and scalable both. Sharding by default has serious security challenges but the "dynamic sharding structure" of Harmony makes it resilient against the attack and makes it a unique scalable blockchain where all the three fundamental attributes (security, scalability & decentralization) are promised to further categorize it as a next generation public blockchain.
It uses effective PoS & runs FBFT algorithm which is linear and scalable. The randomness generation with DRG, VRF, VDF makes the dynamic sharding structure secure against adversaries. The adaptive threshold PoS keep on adjusting the threshold value of the vote share to effectively counter large stake attack. It is resource efficient as the state of blockchain is slimmer in Harmony and it is a state sharded blockchain, so simple resources can also join and easily sync with the network.
Sharding Technology
In general, a blockchain network consists of a number of nodes and these nodes validate the transactions occurring throughout the network. When a node processes/validates a transaction, all other nodes check and verify this and update it in the ledger after checking the correctness of it. The nodes generally keep a record of information and this information includes account balance, transaction, smart contract, etc.
During a time when the network is overloaded with a number of transactions queued up to be processed, the processing gets slower, leading to scaling issues. In order to mitigate this issue, the network is split into a number of small group of nodes and these group of nodes can concurrently process the transactions coming at it, thereby improving the transaction throughput of the overall network. This phenomenon is known as sharding.
Transaction Layer Sharding
The blockchain network is split into a number of group/shards and each shard act as a minor blockchain which is a subset of the whole blockchain. A number of such minor blockchains process the transactions concurrently. All these shards/minor blockchains may process different transactions, but their state remains the same. In other words, any node of the shards, regardless of which shard it belongs to, has the same same state of blockchain as all other nodes of the shards.
State Sharding
The blockchain network is split into a number of shards and each shard act as a as minor blockchain of the whole chain. One may call it a shard chain also. Here also the shard chain processes the transaction which differs from other shards. But then again, there is a fundamental difference exists in State sharding, i.e. the nodes belonging to particular shard contain different state of the blockchain, which is again a subset of the entire state of blockchain. So in state sharding, the shard chains not only process different transaction but also hold a different state as compared to the other shard chains.
That further implies that each shard chain has different account balance, different transactions, different smart contract, etc. But there is a communication facility called "cross-shard communication", using which the user account balance can be moved from one shard chain to other.
The different shards process different set of transactions concurrently across the network and improve the scaling capability of the network. It must be noted that with higher number of shards, the scaling capacity of the network further increases. Harmony leverages on "state sharding" to make the blockchain decentralized and scalable both.
Sharding Structure of Harmony
The blockchain network is split into a number of shards in Harmony and the nodes(called as validator nodes) are assigned into the shards. All the shards possess different state of blockchain and they process different sets of transactions. But the sharding structure of Harmony is a not a static one. Had it been a static one, then it would not be able to counter the varieties of attacks that are generic to sharding mechanism. Therefore Harmony has made its sharding structure dynamic and the sharding structure(i.e. the assignment of nodes in to a shard) varies from one epoch to the other.
An "epoch" is a predefined time interval in Harmony and this nomenclature has been introduced particularly to facilitate the dynamic sharding structure of Harmony. The network runs through one epoch to the next and so on. In each epoch the nodes of the shard changes. That means if in an epoch "e" a node is in a shard "A", then in the next epoch "e+1", it may be assigned to a different shard, say for example shard "B" or "C" or any other shard.
In order to determine which nodes will be assigned to which shard, a random number is generated using DRG, VRF, VDF & based on that number the validator nodes are assigned to the shard. In every epoch, "resharding" is done so as to facilitate the new nodes to come in and join. The design of Harmony in "dynamic sharding structure" & "resharding" makes the network evenly distributed & decentralized.
Beacon chain & Shard chain
The Harmony blockchain is split into a number of groups and these groups consist of a number of nodes which are dynamic in nature as the nodes change in each epoch & assigned to different shards randomly. So these small groups act like a minor blockchain which is a subset of the entire blockchain. It is also called as shard chain.
So a shard chain is the one which processes the transactions that are relevant to it, which hold its own state of blockchain, whose nodes keep on changing in every epoch and it communicates with other shards through efficient cross-shard communication.
A Beacon chain is also similar to shard chain in its structure but in function it has additional functionalities (in addition to the transaction processing like shard chain) i.e.
- It generates random number.
- It accepts the tokens from the participant to join the network & become validators.
Assigning nodes into shards using DRG, VRF, VDF
As Harmony has a dynamic sharding structure, the nodes of a shard keeps on changing in every epoch. The shuffling and assigning of the nodes is based a random number & for that DRG protocol (Distributed Randomness Generation) is used. VRF (Verifiable Random function) and VDF (Verifiable Delay Function) further makes this protocol robust and the assigning of the nodes become more secure against any possible attack.
VRF uses cryptography whereas VDF delays the disclosure of random number to counter last-revealer attack.
The characteristics of the random number used in Harmony are defined as:
(a) Unpredictable- No one should be able to predict it.
(b) Unbiaseable- The process involves in generating the random number should not be biaseable of any participant.
(c) Verifiable- It should be verifiable by any observer.
(d) Scalable- It should be able to scale to a large number of participants.
The random number is generated using DRG protocol with VRF & VDF in the following ways:
- The leader node sends a message with the hash of the previous block to all the validator nodes.
- Then using VRF, a random number is created cryptographically with a proof for each validators. Each validator node sends the (random number & proof) to the leader node.
- The leader node from the set of random number it received, get the final randomness ”pRnd”.
- The leader then runs BFT to reach consensus on ”pRnd” & commit it in the current block.
- The next step proceed to compute the actual randomness , i.e. ”Rnd”. At this step, VDF is used to delay the randomness and this can be disclosed only after a certain number of blocks.
- Once ”Rnd” is computed, BFT is run by the leader node among all validators to agree on the validity and reach consensus.
Synchronization of nodes into the new shard in every epoch
Harmony is a state sharded blockchain which has a dynamic sharding structure. In other words, the nodes keep on changing in every epoch and assigned randomly to new shard.
Technically for any normal blockchain which does not employ state sharding technology, for a node to join the network and to sync with it takes time. The reason being, the new node has to download & store the entire state of blockchain, therefore the synchronization takes time.
As Harmony is a state sharded structure, a new node to join the network & to sync with it, takes less time. The reason being, the new node does not have to store the entire state, instead, the node has to store a subset of the entire state. In other words, the new node has to download the state of the shard chain(to which the new node is joining). That can happen in less time and the new node can quickly sync with the network.
Generally the new nodes download the historical block headers and validates it by checking & verifying the signatures. But then again, signature verification also takes time. Harmony mitigates this issue by adding additional hash pointer to the first block of the last epoch. This makes the verification of the present state of blockchain faster. The new nodes joining the network can jump across the block within an epoch by tracing hash pointers to the genesis block.
Resharding
Resharding is done in Harmony's "dynamic sharding structure" to prevent a malicious actor attacking the fixed shard. Harmony approaches with a unique rule which is known as bounded-cuckoo rule, using which the new nodes joining into the network are assigned to the shards.
(1)It is natural that after the completion of each epoch, some of the validators may unstake, such nodes are evicted from the network and those who keep their stake will stay.
(2) The new nodes that join and stake during the new epoch will get new voting shares.
(3) These voting shares are then randomly assigned to the shards that have more than median of the total voting shares.
(4) Then a constant number of voting shares are derived from all the shards and then they are re-distributed to the other half of the shards which have less than median of the total voting shares.
Cross-shard Communication
In the sharding structure of Harmony, each shard chain holds different state, so different account balance, different smart contract, different transaction. Therefore in order to move the account balance or to run the instance of smart contract it is needed for the shard chain to communicate and interact with other shard chains.
Harmony adopts shard-driven approach to facilitate this and simplify the network cost by using Kademlia protocol, which mainatins a routing table, with the details of routing distance between the nodes. In Kademlia protocol, the message travels across O(logN)) distance instead of O(N). In other words, it can easily trace a node with closest shard ID and the message can travel faster than the normal gossip protocol. In order to further make it efficient & robust, it encodes the communication data with an erasure code and sends the chunks of data to the nodes.
Consensus in Harmony
Harmony uses PoS to allow participants to stake and become validators. Then it runs BFT algorithm to reach consensus. But it has made some improvement over the PBFT, to make the configuration linear and scalable.
In BFT algorithm, there is a leader node and all other are validators nodes. There are two phases also- prepare phase and commit phase. The message broadcast and communication happens between the leader node and the validator nodes. However, in PBFT, it is not linear and the network complexity is O(n*n).
In PBFT the leader broadcast the message to all validators and then all the validators rebroadcast that to all other validator nodes for the purpose of counting the votes. So the network complexity becomes O(n*n) in PBFT.
Harmony has made two improvements over this:
It uses multi-signature signing to collect the votes, so the validators don’t require rebroadcasting.
It uses BLS signatures for multi-signature signing as against schnorr signature, as BLS signature has only one-round trip as opposed to two round trips of schnorr signature.
Harmony with this improvement over PBFT, calls it FBFT(Fast Byzantine Fault Tolerance) and the network complexity in FBFT becomes O(n). So Harmony network with this improvement becomes linear and scalable.
The leader runs FBFT among all the validators & achieve consensus in the following way:
Prepare phase
- The leader constructs the new block and broadcast the block header to all the validator.
- The validators check the validity, sign it using BLS signature and send it back to the leader.
- The leader after having 2f+1 valid signature, including the leader itself, aggregates them, creates a bitmap which indicates which validators have signed & then broadcast it.
Commit phase
- The validators check that the multi-signature has at least 2f+1 signer, verify the transactions in the block broadcasted from the leader in Step 1 of prepare phase, sign the received message of Step 3, and send it back to the leader.
- The leader waits for at least 2f + 1 valid signatures, aggregates them and creates a bitmap logging all the signers. * Finally, the leader commits the new block with all the multi-signatures and bitmaps attached and broadcasts the new block for all validators to commit.
Effective PoS
In a PoS blockchain, the requirement is to stake in order to participate in the network. But it is also a fact that it turns out to yield stake centralization. Big entities can stake more and more and may gain more than 50% control of the network and they may even follow the route by dividing their large stake into small chunks and act as small stackers. So while staking mechanism is core to any PoS blockchain simple staking does not work and that may not make the network decentralized. Harmony uses effective PoS, it is efficient & better than PoS because it prevents centralization and supports stake compunding & delegation. Further it makes the network evenly distributed among the validators.
As it uses effective PoS, the parameter for a validator is "effective stake", which further determines “block reward” and “voting power” and “consensus”.
From the stakes of top ranked elected validators, the median stake is calculated and the actual stake is the amount of the actual stake by the validators.
Effective stake is calculated as:
where c is protocol parameter. The effective stake lies between {(1 + c) * median_stake} & {(1 — c) * median_stake}.
With the introduction of effective stake in Harmony, those who try to concentrate stake in a single validator, are economically punished. In contrast, the low ranked validators get advantage economically from their stake. So "effective stake" helps to make the network evenly distributed & decentralized.
The tokens holders are free to delegate to the validators of their choice based on commission rate, uptime, position in the rank, etc. The delegators can delegate to multiple validators. But they can utilize the effective stake chart to optimize their economic incentive. It is advisable to delegate to the validators in the green area of the chart to earn more as the return to stake ratio is higher. Similarly for compounding, the yellow area is economically incentivized.
There is a slashing mechanism in this effective PoS of Harmony and it punishes for double signing, unavailability etc. The slashing mechanism forces the big stakers to decentralize themselves. In case of double signing, there is a minimum of 2% slashing on stake which increases linearly with the validators being punished & slashed. For unavailability, the voting power leaked by 25% for 3hrs offline, 100% voting power for 12 hrs offline.
Security
A state sharded blockchain is prone to many security challenges but Harmony has designed its structure in such a way that the network is secure against the varities of attack. Tha attackers can not gain access to a dynamic sharding structure of Harmony.
Large stake attack
It is a “single shard take over attack” in which the attacker takes over more than one third of the voting shares.
In order to counter this attack, Harmony:
Applies adaptive threshold PoS which keeps on adjusting the price of a voting share.
Secondly it shards by “voting shares” and not by “validators”. At the start of every epoch, a random permutation on all voting shares is done, after the randomness is revealed. Then the permuted list of voting shares is divided evenly into "m" buckets, where m is the number of shards. The voting shares falling in the i th bucket are assigned to shard “I”. The leader is determined who possess the first voting share in the bucket. The effective PoS on the other hand prevent the large stacker to concentrate their stake.
1% shard attack
It is basically an attack related to a sharded blockchain network with PoW consensus. For example, in a sharded PoW network having 100 shards, it requires 1% of the total network’s hash power for an attacker to completely take over a single shard. This is known as 1% shard attack.
Harmony counters this attack in two ways:
It uses PoS instead of PoW. So hashing power is not a requirement. Now the staking requirement is also not based on actual stake, rather “effective stake”. So the heavy stakers are also discouraged to concentrate their stake at a single shard.
The sharding structure of Harmony is not static, rather it is dynamic which changes in every epoch. The nodes assigned to a shard in an epoch “e”, is not the same in the epoch “e+1”. The random permutation & combination of the nodes and its assignment to shards in every epoch varies and is governed by DRG protocol, VRF, VDF which makes the dynamic sharding structure unpredictable.
Replay attack
In this type of attack the old transactions are re-submitted in the deleted account, that is why in Etherum blockchain, a particular nonce with empty account can not be deleted. But because of this, the available space in the state of blockchain remains empty. So in a chain, where empty account can not be deleted, can not be optimized and utilized to its full potential.
Harmony counters this attack by letting the transactions specify the hash of the current block. In other words a transaction is only valid before a certain number of blocks following the block of the specified hash. So in Harmony the empty accounts can be deleted and the space in the blockchain can be optimized and utilized fully. As a result, the state of the blockchain also remains slim in Harmony.
Merits of Harmony
- The dynamic sharding structure of Harmony is most secured.
- The empty accounts in Harmony can be safely deleted.
- The ”effective stake” concept decentralizes and evenly balances the network and prevents stake centralization.
- Harmony is very effective to counter various security challenges such as: large stake attack, 1% shard attack, replay attack, etc.
- Harmony keeps all the three fundamental attributes intact- security, decentralization, scalability.
- The scaling capacity is significant- 118200 tps with 44000 nodes.
- FBFT makes Harmony linear and scalable.
- Harmony is a potential blockchain infrastructure for the next generation applications, large economy, gaming, data trading, etc.
- The state of Harmony blockchain is slimmer, optimized & enables simplified communication between the nodes.
- New nodes joining the network can quickly sync with the Harmony network as the new node has to download the state of the shard chain (to which it is joining) only which is a subset of entire chain.
In this use case "Play One" wants a scalable platform and a network which can take heavy load, the players should be able to trade NFT tokens seamlessly.
Harmony leverages on state sharding technology and brings innovation at every layer to make it scalable. The testnet result has been 118200 tps with 44000 nodes. It is also cost effective. So transaction fees will be definitely low.
Most importantly, it does not use chain-based consesus, it runs through FBFT, uses BLS signature, which is linear and really fast as compared any PoW based consensus chain. The communication complexity in its network is also linear and state of the network is also slimmer as compared to other existing blockchains. That implies Harmony will not get clogged with heavy load and it can seamlessly handle large number of players.
"PlayOne" enterprise can build its online gaming platform on the top of highly scalable Harmony platform and let its players enjoy the gaming platform with low transaction cost, smooth network without any congestion even during peak load, fast transaction processing. Overall the gaming experience will be a lot of fun for players.
So "Play One" finally decided to join the Harmony network to build its online gaming platform.
Evaluation & Rating
Harmony is a promising project. If the blockchain technology has to become mainstream, then the approach of a blockchain infrastructure should be like what Harmony has offered. The state sharding has really opened up new possibilities which has not achieved till date.
I have evaluated Harmony based upon the various parameters as shown below and I would like to give a rating of 4.8 star out of 5 stars to Harmony.
Conclusion
Harmony as a public blockchain has enormous potential. The fact that state sharding has never been done before by anyone and recognizing the fact that sharding by default has serious security challenges & still taking this bold step and mitigating all the security issues promptly is something like a bright sunshine in the domain of blockchain technology.
The way the dynamic sharding structure has been arranged and designed is really spot on for any blockchain enthusiast who is looking for an optimized and dynamic blockchain where the applications can enjoy high scalability, security and decentralization altogether. The testnet scalability report is a testiomony to its claim of a Blockchain for Billions. However, as it progress with the mainnet, it may encounter a number of real challenges, but with the dedicated & passionate team behind Harmony, I hope that should not be a problem and Harmony as a provably secure, decentralized, scalable blockchain will certainly grow big to become the public blockchain for Billions in reality.
References
- Harmony website
- Harmony OnePage
- Harmony Whitepaper
- Harmony Medium
- Harmony Twitter
- Harmony LinkedIn
- Harmony Instagram