ZebraCrossing Response To Quarkchain
This is a response to Quarkchain's article "Response to the article "Quarkchain Red Flags" - We Know Something You Don't Know" [1]. My original article "Quarkchain Red Flags - We Know Something You Don't Know!" [2] is given in Reference.
All my previous query and comments in article [2] appeared as italics.
“If one of the shard in state sharding is compromised, then the tokens stored in that shard might be lost forever and could not be recovered. This is called a single shard takeover attack and the same has been discussed in Ethereum FAQ. In transaction sharding, recover is possible since each shard store a complete global ledger.”
Quarkchain Response: We respectfully disagree with the claim here. First, a shard does not mean a node. A shard is a blockchain run by many many nodes from different clusters and each node in a cluster has a copy of the full ledger of that shard. Please see QuarkChain whitepaper Figure 7 (a) and (b) for a detailed example. In addition, Ethereum sharding is also working toward state sharding direction.
ZebraCrossing Response: Please refer the Ethereum FAQ about single shard over attack. Let me Quote it from Ethereum FAQ here. “ Single-shard takeover attacks - what if an attacker takes over the majority of the collators in one single shard, either to prevent any collations from getting enough signatures or, worse, to submit collations that are invalid?”. Nowhere in the article, I have mentioned that node is a single shard and you made an assumption that I am talking about a single node. I am well aware that a shard can comprise of multiple nodes and this is the exact attack which has been mentioned in Ethereum FAQ as well. From your white paper you mean to say “Clustering with Honest Nodes” and yet you clearly don’t specify what happens if the clustering nodes were both taken over. Why do you think State sharding is not achieved in ethereum? If the single shard takeover attack is executed successfully, the states that are sharded in the particular shard will be gone and your balance will be unusable. Another quote from an interesting article published recently. “A solution to this problem is to maintain archival or backup nodes that can help the network troubleshoot and recover from data unavailability. However, those nodes will then have to store the entire state of the system and hence may introduce centralization risks.
Another point to consider in any sharding mechanism (certainly not specific to state sharding) is to ensure that shards are not static for resilience against attacks and failures; the network must accept new nodes and assign them in a random manner to different shards. In other words, the network must get reshuffled once in a while.
However, reshuffling in the case of state sharding is tricky. Since each shard only maintains a portion of the state, reshuffling the network in one go may render the entire system unavailable until some synchronization is completed. To prevent outage, the network must be reshuffled gradually to ensure that every shard has enough old nodes before a node is evicted.
Similarly, once a new node joins a shard, one has to ensure that the node is given ample time to sync with the state of the shard; otherwise the incoming node will reject outright every single transaction.” However Quarkchain (your) white paper mentions nothing about any of the challenges and claim you solve this using clustering.
“It is also unclear how reshuffling can happen in state sharding, and will that temporarily makes the network unusable? These questions were asked in Quarkchain main channel and no answers were given by their admin. It seems like the team at Quarkchain has no idea how to tackle the problems in state sharding and yet dare to claim it will use state sharding to achieve 1 m tps!”
Quarkchain Response: Reshuffling or resharding is an interesting topic. We will reveal the technical details later. In short, the design is very similar to Google’s BigTable’s resharding -- splitting the state of a shard so that one could have two shards. Based on Google’s pioneer work, this can be done efficiently.
ZebraCrossing Rebuttal: Sharding in traditional database has been done for many years. Doing the same in Distributed systems is the challenge. What you are referring to the big table is not actually sharding but rather distributed storage proposed in the below white paper.
https://static.googleusercontent.com/media/research.google.com/en//archive/bigtable-osdi06.pdf and “. Based on Google’s pioneer work, this can be done efficiently” if this would have done efficiently, it would have been solved 3 years ago and ethereum wouldn’t be scratching their heads still now.
“Do note the team from Quarkchain are comprised from Phd and Professors from the field of Electrical and Electronics Engineering, not from blockchain related field and has zero blockchain experience prior to this. It is questionable that the team can achieve what they claim of 1 m tps while still maintaining sufficient security and up time.”
Quarkchain Response: Blockchain itself is not a brand brand new topic. In fact, it highly relies on many existing areas, e.g., security, distributed database, distributed network, parallel computing, signal processing, optimization, network theory, big data, etc. In the near future, it also related to hardware circuit designs. QuarkChain team actually accumulated tons of experiences and various expertise in many of these areas from academia to industry. Our core engineers previously worked at Google and Facebook -- two companies that have been successfully using sharding technology in centralized world for years. The responsibility of our core engineers was using sharding to solve the scalability problem of large distributed systems. Thus, our team is very confident about our project.
ZebraCrossing Rebuttal:
If you do really have an experience in large scale distributed systems, you wouldn’t have put the ridiculous numbers such as 1 million TPS and changed it to 100K TPS.
“Why Quarkchain still using PoW rather than the recently more popular PBFT algorithm? It is because the limitation of PBFT algorithm that the number of nodes that PBFT can support is very limited (not more than 50 nodes). Zilliqa is using PBFT+PoW, a hybrid and improved consensus algorithm. It is clear that what Quarkchain team try to accomplish is rather trivial. They don’t have the technical know how on how to make a robust and secured sharding blockchain. They have no idea how to handle the issues of state sharding. They don’t even realize that by increasing the number of shards, their actual TPS will reduce due to increasing cross-shard transactions as explained below in Cross-Shard Transactions.”
Quarkchain Response: QuarkChain is using PoW for its consensus protocol for shards. “Why not PBFT?” Nobody defined PBFT is more popular. In fact, those representative ones like Bitcoin and Ethereum are all using PoW. “What QuarkChain team try to accomplish is rather trivial” -- Does this mean “scalability is trivial”? We think the majority of blockchain society won’t agree with that. As stated in the previous response, our core developers have years of experience working on solving scalability problems in large scale distributed systems in Google and Facebook. How do the author conclude that “ they have no idea how to handle the issues of state sharding”? About the “TPS will reduce due to increasing cross-shard transactions”, we will response in the following.
ZebraCrossing Response: Can the team share what are the projects they have worked in “large scaled distributed systems in Google and Facebook” ?
Another issue of the Quarkchain is no minimum and maximum nodes per shard, see Figure 1. This will jeopardize the security of the blockchain. If a shard with low nodes count, it is easy to be targeted by hacker to compromise that shard and lost the tokens. Although the root chain will record each shard hash, it does not record a copy of the shard’s ledger. The ledger is local to each shard only since shard do not share its ledger. The root chain can only identify a genuine shard vs a compromised shard, as Figure 4 shows.”
Quarkchain Response: This comment actually proves the misunderstanding of the author to state sharding technique. Therefore, the plot given by the author does NOT represent the structure of QuarkChain network. QuarkChain supports clusters with multiple nodes which provide security. Please see QuarkChain Whitepaper Section 5.1 and Figure 7 (a) and (b) for a detailed example.
ZebraCrossing Response: Please know that the question is valid since the clusters can be compromised, you didn’t explain how clusters are formed in the white paper and you didn’t explain what happens when the clusters are taken over by byzantine nodes and security threat model is totally unexplained and hence the question still stands.
It is also said that a 25% hashpower attack will compromise the whole network of Quarkchain. This is a major red flag and shows how insecure the architecture of the Quarkchain is, see Figure 5.”
Quarkchain Response: All transactions in the QuarkChain Network are protected by 50% of the overall hash power of the network, and a double-spend attack requires at least 25% hash power. This is smaller than single-blockchain’s 50%, but since the QuarkChain Network is more decentralized, a miner will be much harder to collect 51% hash power in our network than that of single-blockchain. More details about decentralization and security, please refer to white paper Section 4.
ZebraCrossing Response: How is it more decentralized when the miners can choose to mine their own shard ? Again, you guys are misleading. A Miner doesn’t have to collect 51% of the hash power to attack your network. All he need to do is get more than 25% of the hash power to attack the root chain and that’s it the entire system is compromised.
“It is also said that the address of each wallet last few digits are shard ID, from live test video [7]. This will give an easy target for hackers to target those wallets with large amount of tokens and knowing that which shard the hacker need to compromise. Thus the design of Quarkchain architecture is simply a big warning.”
Quarkchain Response: Again, the author still messed up the concepts of shards and nodes. Every cluster would main the full ledger of all shards, hacker must compromise the whole network if a shard is compromised? In addition, all transactions in the QuarkChain Network are protected by 50% of the overall hash power of the network and all shards are supported by clusters. Please refer QuarkChain Whitepaper Sections 4 and 5.
ZebraCrossing Response: Again your comments are totally misleading. What my comments meant was if I know which shard I am going to be in based on shard ID, I can easily target the nodes within the shard and falsify transactions. Again you are talking about clusters. But clusters are nothing but grouping of shards / nodes. So, the same security model still applies there.
“Which means a Cross-shard transaction will be solely confirmed by a root chain at the end of each block (which is 15 times slower than the shard chain) [6]. This is an interesting side-effect of the sharding that you might be unlucky to have your transaction pending for 15x time than someone else because you are doing a cross-shard transaction [5].”
Quarkchain Response: It is true that receiving a cross-shard transaction needs root chain’s confirmation (by confirming the transaction’s block header), which takes longer time than in-shard. However, when initiating a transaction, a user could recognize whether they are performing cross-shard transaction or not based on the address info. In term of finality, both in-shard transaction and cross-shard needs the same amount of root chain’s confirmation.
“How do the smart contract keep track of the number of total tokens sold? The maximum number of tokens in this case is 1 million tokens. User can buy from any shards.”
“In other to keep track of the total of tokens sold, few approaches can be used.
A) Cross-shard communication. Each smart contract from each shard will update each other if a token is sold. If there is one million shards, that means if one token is sold, 1 million of communications via the root chain is needed! It means It is VERY RIDICULOUS SLOW! OR
B) A special smart contract say in Shard 5 is used specially to keep track of the tokens sold. So, whenever a shard sold a token, it need to update special smart contract in Shard 5, much faster but still a lot of overhead, the problem of 5 times more fees still haven’t solved. OR
C) Divide the total tokens by number of shards. 1 m tokens here divide by 5 shards, so each max tokens available for sold in a shard is limited to 200,000 tokens only. This method will eliminate cross-shard communication and most efficient, but remain costly to launch so many smart contracts at the same time. But it will at the expense of user inconvenience. If shard 1 has a lot of participants, then it sold out way faster than shard 2 which is unfair to participants in Shard1.”
Quarkchain Response: We feel that the arguments here are all based on the wrong concepts of state sharding and cross-shard transaction. The QuarkChain Network will support smart contracts via Ethereum virtual machine (EVM). To utilize high-scalability feature of the QuarkChain Network, an additional scalability-aware interface will be provided with features such as which shard the contracts being executed and sending smart contract specific data via different shards. Please also refer to our Whitepaper Section 6.4 on how smart wallet supports the smart contract for cross-shard transactions.
ZebraCrossing Response: Lol, your white paper doesn’t provide any details on how do you handle cross shard communication when state of account is updated by smart contract. You do know, when state of the account is updated by smart contract that has to be communicated with all the shards right? Even factoring in with your clustering, you would have to communicate the account balance to all the clusters. No details on this yet. Yes, I am aware that you will use EVM to support ethereum smart contracts. Define “Scalability aware” interface? Again this is all words and no details on this.
“What about Dapps running in the future in Quarkchain? A Dapps cannot run on all shards since each shard is like an island of itself, with its own ledger and own smart contract execution independent of each other. The only bridge to connect other shards is via the root chain, but the root chain do not record transaction details and has no copy of the shard’s ledger and definitely will not process smart contract.”
Quarkchain Response: This is a misunderstanding. Based on the author’s plot -- the “star” shape shard, the shard may look like an island. However, that is NOT our definition of shard. Apparently the author got confused about transaction and confirmation. The cross-shard transaction needs root chain to confirm. However, thanks to the sharding technique, the transaction finality of QuarkChain (no matter of cross-shard and in-shard) takes the same time, and the network throughput is still high. User end is barely recognizing the transaction finality difference between cross-shard and in-shard transactions. The QuarkChain Network will support smart contracts via Ethereum virtual machine (EVM). To utilize high-scalability feature of the QuarkChain Network, an additional scalability-aware interface will be provided with planned features such as which shard the contracts being executed and sending smart contract specific data via different shards. Please also refer to our Whitepaper Section 6.4 on how smart wallet supports the smart contract for cross-shard transactions.
ZebraCrossing Response: Again you are misleading. I am not talking about finality of the transaction, but the over head of cross shard communication. The worst case overhead for your cross shard communication could be O(n ^ k) where k is the number of shards. Your white paper mentions that cross shard communication should increase TPS. But on the other hand in a sharded system, the cross shard communication should be kept as minimal as possible; otherwise the overhead would bring the entire network down which would be the case in your architecture as well. Again , nothing on your white paper about this overhead or what is the best, worst and optimal time taken for overhead scenarios.
References