A complete explanation of blockchain scaling solutions that you can understand at first glance

authorship

Chasey | Buidler DAO Researcher

-WeChat: Chasey1021

*This article is for personal analysis only and does not constitute any investment advice. If any incorrect content appears, please point it out

Every blockchain faces a causeThe impossible triangle of decentralization, security, and scalability. Among them, decentralization is the biggest advantage of blockchain technology and requires priority protection; if we want to establish a long-term, sustainable ecosystem, security is also an element that must be achieved to the extreme. This causedCurrently, public chains generally have poor scalabilityThe current state of affairs.

Source: When I ate lotus root like this, I drew it myself

How to improve blockchain throughput = scaling on the premise of balancing decentralization and security is a problem that needs to be solved urgently. In recent years, ETH 2.0, as Ethereum's expansion vision, has received much attention and anticipation around the world, even after many ticket hops. It can also be seen from this that capacity expansion has become a group demand for public chain users, and throughput is also one of the essential indicators when analyzing and valuing a blockchain. The purpose of this article is to provide an overall picture of the current blockchain expansion plan to help readers understand the basic concept of the expansion plan more clearly.

8000 words in full text, estimated reading time 20 minutes

Quick overview of the article:

01/ Why do you need to expand

02/ Classification of expansion plans

02/01/ On-chain expansion

02/01/01 Consensus Layer = BFT; Satoshi Nakamoto; Hybrid

02/01/02 Data layer = block expansion; data reduction; DAG

02/01/03 Network layer = sharding

02/02/ Off-chain expansion

02/02/01 Status channel

02/02/02 Off-chain calculation

02/02/03 Backlinks

03/ Summary and future outlook

04/ References

Before we start discussing the specific expansion plan, let's sort it outThe role and necessity of expansion。

The nodes on the blockchain are divided into full nodes and light nodes. In order to ensure the integrity and security of transaction data, all nodes need to store transaction data for the entire blockchain; light nodes only need to store block headers and verify transactions by requesting the corresponding body from all nodes. The higher the number of nodes, the more decentralized the chain is, and the more work it needs to handle to reach consensus, the greater the negative impact on throughput. In addition, as shown in the figure, the maximum block size of Bitcoin is 1 MB, while Ethereum has set a gas limit (to prevent DDoS attacks), so the upper limit of the block size is limited to about 130 KB.

Source: Blockchair

Since the block size is limited, miners are unable to package all transactions in the same block, so they prefer to rank transactions by how much they expect (Gas Price) and selectively package them by price from highest to lowest to ensure the highest return. This causes transactions with lower gas prices to be delayed too long. As shown in the figure below,About 170,000 transactions are awaiting verification every minute on Ethereum.

Source: Etherscan

Currently, Bitcoin's throughput is as low as 7 TPS (transaction-per-second), while Ethereum's throughput is limited to 15 to 20 TPS. To make it easier to understand, let's compare this with the throughput of traditional online transaction methods: PayPal's transaction processing speed is around 200 TPS, while VISA is about 1,700 TPS, which is quite a big difference.

Additionally, increasing transaction data is putting pressure on the storage capacity required to maintain the blockchain.Currently, Bitcoin's storage capacity has exceeded 400GB, yoy 17.4%; while Ethereum is nearly 900GB, with an average growth rate of 64.30%.

Source: Blockchair

As shown in the figure,More than 1,250,000 transactions occur on Ethereum every dayAlso, with the gradual spread of the public chain ecosystem, this number will increase in the future, and the pressure on throughput will also increase, so there is no time to delay capacity expansion.

Source: yChart

After reading this, we've learned the importance of scaling up, so it's time to take a look at what methods can actually be used to achieve expansion.

The image below is an illustration from Handbook of Research on Blockchain Technology (2020). In this article, we'll look at the “Write Performance” section of the diagram, starting withOn-chain and off-chain anglesExplain the expansion plans that have already appeared.

Source: Handbook of Research on Blockchain Technology (2020)

An on-chain scaling scheme refers to the adoption ofPlan to change the design of the original chain to achieve the expansion effect. Blockchain technology can be split intoSix hierarchical structures: Consensus layer, network layer, data layer, incentive layer, contract layer, application layer. The first three of these are the underlying foundations of blockchain, and they are also the goals of on-chain scaling solutions.

1. Consensus layer = BFT; Satoshi Nakamoto; hybrid

The consensus mechanism refers to the blockchainThe process by which various nodes agree on the availability of data and the consistency of the ledger state.Since the consensus mechanism completely determines the entire process from downloading data to packaging and generating blocks, the efficiency of node verification of transactions depends heavily on the design of the consensus mechanism. The current mainstream consensus mechanism can be divided into BFT consensus, Satoshi Nakamoto consensus, and mixed consensus.

• BFT consensus

  Referring to the BFT calculation (Byzantine Fault-Fault), I need to first talk about the Byzantine generals problem: the Byzantine Empire (Eastern Roman Empire) was committed to expanding its territory. In a war, they tried to send 10 separate armies to encircle the enemy, which could withstand up to 5 Byzantine armies. Since each army stands far apart, the generals of each army need to reach a consensus on action by sending attack/retreat signals to other armies (counting themselves, if they receive 6 or more offensive signals, they will attack; if the opposite is true, retreat). The biggest problem facing generals is: what if a traitor appears in an army and deliberately sends the wrong signal? In blockchain, the problem is similar: Each node in the blockchain reaches a consensus by sending information to other nodes. What if a traitor node (Byzantine node) appears in the network and sends the wrong information?

  
  

  The most famous BFT consensus is PBFT. If you are interested, you can check the references [17] for yourself. The expression is very simple and easy to understand, so I won't go into detail here. Using this kind of consensus, it is necessary to ensure that each working node uses the same random number and block algorithm to calculate and generate blocks. When the original ledger is the same, the calculation results are the same, and the generated ledger cannot be tampered with and permanently disclosed. Since every node needs to synchronize consensus with all other nodes,When the number of nodes is small, it is possible to achieve extremely high throughput while ensuring security. As the number of nodes increases, the amount of data to be processed will also increase accordingly, causing the speed of processing transactions to drop dramatically.

  
  

• Satoshi Nakamoto consensus

  The Satoshi Nakamoto consensus mainly includes Proof of Work Proof-of-Work and Proof of Stake Stake. In this section, we'll attach DPoS, a variant of PoS, to explore the performance of each consensus mechanism in terms of throughput.

  PoW：Computing power determines bookkeeping rights(It is also the right to vote), and there is no need to set up a mechanism to authorize nodes. In terms of throughput, its biggest problem isSlow block generation due to high difficulty; Also, in order to ensure the consistency of the ledger, it is also necessary to intentionally set a packaging delay. The delay here means that after the block is packed, the miner needs to prove the workload of at least one more block before the candidate block can be confirmed.

  
  

  PoS：Hold coins to obtain bookkeeping rights and voting rights(Separated), the packaging node packages the candidate blocks and broadcasts, and the voting nodes vote on the candidate blocks to decide whether to join the blockchain. The voting uses a majority system. Compared to PoW, PoS sacrifices part of security due to the introduction of a voting mechanism; in terms of throughput,Delays are extremely low due to fast packaging and no waiting times.

  DPoS: Holds coins to obtain voting rights, and elects a board of directors to keep accounts.Part of the degree of decentralization was sacrificed on top of PoS, has higher throughput than PoS.

  
  

• Hybrid consensus

  As the name suggests, mixed consensus refers toConsensus that combines the advantages of different consensus mechanisms. Examples include using PoW on the main chain to ensure security while using PoS on the sidechain to guarantee throughput; combining PoS and PBFT, the number of subsequent nodes is reduced to a constant value, thereby increasing throughput again, etc.

2. Data layer = block expansion; data reduction; DAG

In addition to the consensus mechanism,The number of transactions that can be packed in each block is also closely related to the transaction's throughput.We can increase blockchain throughput by expanding block capacity and reducing transaction data, or directly use DAG data structures to process transactions.

• Relaxing/Removing Block Size Limits

  Expanding the block capacity allows more transaction data to be packed into each block, but at the same time, it will increase the time for block broadcasts and increaseNetwork delays, which increases the risk of hard forks.

• Shrink the data stored in blocks

  Among such solutions, the most famous one is Segwit Segregated Witness: the signed part of the block information and the data used to calculate the transaction ID are managed separately, thusCompress the volume of transaction information by 60%. It is effective as an auxiliary solution to the capacity problem, but it does not solve the essential problem.

• Directed Acyclic Graph (DAG)

  As shown in the following figure, the blockchain uses a chain structure, and the block header can only contain the hash value of one block; the block header under the DAG structure can contain the hash value of multiple blocks; new blocks in the blockchain will be added to the end of the chain and cannot be continued from the middle of the chain; the DAG can be rewritten from the previous block.

Source: Russian Blogs

Blockchain is synchronous accounting, and nodes need to record the same information at the same time;DAG is asynchronous accounting, and different nodes can record different information at the same time.As a result, the DAG can package more transactions per unit time, and the TPS is extremely high. Currently, there are two main DAG-based protocols: SPECTRE and PHANTOM.

The SPECTRE protocol defends against attacks through voting

As shown in the following figure, when the content recorded in block X and block Y collide, blocks 6 to 8 recorded after block X will record the same information X; blocks 9 to 11 will record the same information as block Y; block 12 can also go back to block X and block Y at the same time, so it will record the same results as the previous round of voting (in a dotted line), that is, X; blocks 1 to 5 will vote based on the voting status of the block that records its own block information. Since there are more blocks recorded for X, 1 to 5 will select 1 to 5 to select the same information as block 1 to 5 Vote for block X. Since malicious blocks won't be associated with honest blocks until the attack begins, when using the SPECTRE protocol, as long as there are more honest nodes, conflicting transactions can be excluded. The problem is that it only applies to general transactions. Since there is no way to linearly sort all transactions according to time, smart contracts cannot be run. The PHANTOM protocol solves this problem.

Source: An Overview of SPECTRE

The PHANTOM Council will first select honest blocks through voting and topologically rank the honest blocks

Before understanding the screening method, you need to first understand the bifurcation coefficient k = the number of forks that can be forked (for example, forks are not allowed in blockchain, so k is 0) and the GHOSTDAG algorithm = by tracing historical blocks, the longest chain is selected as the main chain to form subset S. By default, the blocks in this set are all honest blocks. Thereafter, for each block, verify whether the intersection of the unrelated block and subset S is less than or equal to k. If less than or equal to k, it is judged to be an honest block and added to subset S.

Source: An Overview of PHANTOM

The following figure (k=3) shows an example: Assuming we now need to determine the authenticity of block I, the blocks derived from block I are blocks K, M, O, P, and R, and the blocks that can be traced back through block I are blocks C, D, and the original blocks, respectively. At this point, the blocks that have nothing to do with block I are blocks B, E, F, H, J, L, N, Q, S, T, and U. The ones that intersect with subset S are blocks B, F, and J, which are equal to k, so block I is judged to be an honest block.

Source: PHANTOM: A Scalable BlockDAG Protocol

The sorting method uses topological sorting: first, blocks with no traceable blocks are used as original block 0, then blocks without traceable blocks other than block 0 are selected as block 1, and so on.

Source: Kappo's Blog

Currently usedThe DAG structure has a high degree of project centralization, so there won't be a more in-depth discussion here. Families interested in DAG can search DAGLabs to learn.

3. Network layer = sharding

Sharding refers toThe ledger is split into several parts, each managed by different node groups.By implementing state sharding,Each node needs to process less transaction dataNot only can it improve transaction processing speed, but the performance requirements for nodes will also be relatively reduced, reducing the threshold for participating in mining and increasing the degree of decentralization.

Source: Why Sharding Is Great: Demystifying the Technical Properties

An off-chain scaling scheme refers to the premise that the original chain structure is not changed,By processing transactions outside of the main network, processing pressure on the main network is reduced.There are three main types: stateful channels, off-chain computation, and multi-chain. For easy understanding, we have grouped all sidechains and sub-chains into multi-chain solutions.

1. Status channel

   
   

A state channel refers to locking a portion of the blockchain state between specific participants through multiple signatures, etc. (opening a channel) -> responding to state transitions that occur in the channel, updating the status under the chain -> confirming the final state with the consent of all participants, and broadcasting it to the main chain.Since only the final state is broadcast, using a state channel to handle trivial mutual transactions can effectively reduce the number of transactions broadcast on the main chain and reduce transaction delays.Additionally, each participant can interact with other participants who haven't opened a channel through an intermediary: Alice and Bob have a channel, and Bob and Carol have a channel, so Alice can interact with Carol through Bob without opening another channel. The transparency of the status channel is low,It usually only applies to frequent transactions that occur between specific participants.

Source: EthHub

2. Off-chain computing

Off-chain computing aims to increase on-chain throughput by moving all functions other than verification off-chain. It mainly needs to ensure security and privacy. The specific operation methods can be divided into four types: verifiable off-chain computing, “enclave” off-chain computing, off-chain secure multi-party computing, and motivation-driven off-chain computing.

Source: Blockchain-Based Reputation Systems: Implementation Challenges and Challenges

• Verifiable off-chain computation: zk-SNARKs, Bulletproofs, zk-Starks

  The off-chain prover uploads the off-chain calculation results to the chain, and the on-chain validator checks them.

  
  

• “Enclave” off-chain computing: Enigma, Ekiden

  A trusted execution environment (TEE) is created on the blockchain node and the data interface is preset for calculation. The TEE acts as a black box, which can effectively protect data privacy while implementing plaintext data computation and improving computational efficiency.

  
  

• Off-chain secure multi-party computing

  The data is split and distributed to each node. The nodes can obtain complete calculated data by combining the data calculated by each node according to the current state of the blockchain and changes in the calculation state of the data obtained by each node. Nodes require less data to compute and are more efficient.

  
  

• Incentive-driven off-chain computing

  The solver calculates the transaction data and pledges a security deposit at the same time as publishing the results; the validator verifies the solver's results, and if an error is found, it can pledge the security deposit and initiate on-chain arbitration, and the correct party will receive the processing fee paid by the user.

  
  

3. Outer chain = side chain; rollup

• side chain

  A sidechain is a blockchain completely independent of the main chain. It uses lock+mint/destroy methods to project assets from the main chain to the sidechain, and completely complete the entire process of transaction processing and storage on the sidechain.The security of a sidechain depends entirely on its own nodes and consensus mechanism.There are two types: anchored sidechains and federated sidechains (adding multiple signature addresses between the main chain and sidechain to verify transactions to reduce delays).

  
  

Source: EthHub

• Rollup

  The difference between Rollup and sidechains is that it only processes transactions on the sub-chain, and the data is still stored on the main network, so it canEnjoy data security on the main network while improving transaction processing efficiency.Rollup can be divided into four types according to data availability and transaction verification methods:

  
  

  Proof of fraud (report invalid transactions) + off-chain DA (security-, scaling effect+) = Plasma

  
  

  Plasma needs to create a smart contract with the child chain hash state transition rules written on the main chain to connect the main chain to the child chain (and grandson chain, grandson chain...). Its expansion mechanism is similar to the state channel.Increase throughput by reducing transactions that need to be processed and saved on the main chainHowever, it is not a channel between specific participants that is established outside of the main chain, but a new blockchain with an independent consensus mechanism for shared use. The sub-chain needs to regularly submit status updates to smart contracts on the main chain, and it passes smoothly7-day questioning period(Optimistic proof, to ensure transaction security) The block state is then written to the main chain. It combines the advantages of state channels and sidechains: in the state channel mode, if new participants need to be added, a new channel needs to be reopened on the chain, and not in Plasma; the state channel requires the consent of all participants when synchronizing the state to the main chain, and Plasma does not need; the state channel only holds the final state, and Plasma's state transition records have a complete record on the sub-chain; state transitions on the sidechain directly affect the state of the main chain, so the security of its assets depends on the sidechain itself, and Plasma is not Security depends on the main chain.

  
  

  The main problems faced by the Plasma mechanism are:A child chain node needs to keep a large amount of transaction data on the child chain; the node must be online.

Source: Plasma: An Innovative Framework to Scale Ethereum

Validity proof (inference transaction is valid) + on-chain DA (security +, expansion effect-) = zk-rollup

zkRollup can effectively improve the problems faced by Plasma. Since zkRollup and Validium in this section both use zero-knowledge proof as a verification mechanism, let's first briefly explain the concept of zero-knowledge proof in this section: zero-knowledge proof means that when proving to others that a proposition is true, no other information is provided to support it other than that the proposition is true. For example, A uses zero-knowledge proof to prove to B that he is an adult. At this point, B won't know A's birthday; they will only know that A is an adult. In zkRollup, zkSNARK is used to verify a large number of transactions, and after determining the validity of these transactions,All you need to do is upload a zero-knowledge proof that these transactions are valid on the main network.This greatly reduces the amount of data, and allows transaction data to be written to the main network.

zkRollup is also facing a few issues:Zero-knowledge proves difficult to calculate; requires initial trust; poor versatility

Validity proof (inference transaction valid) + off-chain DA (security-, expansion effect +) = Validium

Validium also uses zero-knowledge proofs to guarantee the validity of its transaction information and the availability of off-chain data. The only difference between it and zkRollup isBringing data availability off-chain allows Validium to have higher throughput, but the consequence is that data availability managers can modify Merklized status slightly, making it impossible for users to transfer funds.As shown below, if d3 is modified, the owner of d1 will not be able to obtain information about node m, and m is a necessary condition to prove ownership of their account. After zkRollup came out, Validium basically completely lost its competitiveness.

Source: Validium And The Layer 2

Proof of fraud (reporting invalid transactions) + on-chain DA (security +, scaling effect-) = Optimistic Rollup

Optimistic Rollup is an upgraded version of Plasma, and also addresses zkRollup's generality issues. The transaction information submitted by its default node is correct. After submitting the transaction information to the main network, there will be7-day questioning period, for others to check the correctness of the transaction. It differs from Plasma by writing transaction data onto the chain; unlike zkRollup, it does not use zero-knowledge proofs. While neutralizing the other two types of Rollup, Optimistic Rollup alsoPart of the throughput was sacrificed.

Source: ethereum.org

Because of the existence of an impossible triangle, there is bound to be no perfect expansion plan, and the costs of each plan need to be weighed. I personally think that since on-chain scaling is more expensive (including causing hard forks and technical difficulties), it is more difficult to achieve. Under normal circumstances, off-chain scaling solutions will still be the main focus.

In the current off-chain scaling scheme, Rollups maintain an advantage due to its security.However, although L1 throughput can be greatly released through Rollups, the speed at which L2 packed data is transferred back to L1 is still limited by the block size of Ethereum. Currently, a block of Ethereum is stored at around 100kb. Calculated, the Rollups package data that can be processed within a minute is less than 500kb. In addition to receiving the packaged data, the main network also needs to process the original transaction information on L1, which has led to a bottleneck in the throughput of the off-chain scaling scheme.

But the good news is that it's OKCombining different solutions: In the on-chain solution, the idea of DankSharding can be used to drastically reduce the verification node's workload and data download volume by using sample verification and RS coding to solve the problem of transaction processing speed. It is expected that it can complement Rollup after implementation. However, if the implementation of sharding is taken as a major premise, the cost of Optimistic Rollup is mainly the cost of writing into the main network, and is most suitable for ProtoDankSharding, which can reduce this cost, so I personally think that in the near future (if EIP-4844 is successfully implemented), off-chain Optimistic Rollup+ on-chain ProtoDankSharding would be the best choice.

treatises

1. Yonatan Sompolinsky & Shai Wyborski & Aviv Zohar (2021). PHANTOM GHOSTDAG A SCALABLE GENERALIZATION OF NAKAMOTO CONSENSUS

2. Dodo Khan & Low Tang Jung & Manzoor Ahmed Hashmani (2021). Exploring Literature Review of Challenges in Blockchain. Applied Sciences 11 (20)

3. Ammar Battah & Youssef Iraqi & Ernesto Damiani (2021). Blockchain-Based Reputation Systems: Implementation Challenges and Challenges. Electronics 10 (3)

4. Saravanan Krishnan & Valentina E. Balas &... & Raghvendra Kumar (2020) Handbook of Research on Blockchain Technology. Academic Press

5. Yaqin Wu & Pengxin Song & Fuxin Wang (2020). Hybrid Consensus Algorithm Optimization: A Proposed Method Based on POS and PBFT and Its Application in Blockchain

6. Yonatan Sompolinsky & Aviv Zohar (2018). PHANTOM: A Scalable BlockDAG Protocol. IACR Cryptol

books

7. Brian M. (2018). The Essential Guide to Bitcoin Mining: A Cryptocurrency Tutorial. Holmes House Press

websites

8. Vitalik Buterin (2021). The Limits to Blockchain Limits. https://vitalik.ca/general/2021/05/23/scaling.html

9. Vitalik Buterin (2021). Why sharding is great: demystifying the technical properties. https://vitalik.ca/general/2021/04/07/sharding.html

10. Vitalik Buterin (2021). An Incomplete Guide to Rollups. https://vitalik.ca/general/2021/01/05/rollup.html

11. APTX-486977 (2021). Ten thousand character long article interprets the seven types of blockchain consensus algorithms. https://bbs.huaweicloud.com/blogs/242691

12. Build Blockchain Tech (2020). Validium And The Layer 2 Two-By-Two — Issue No. 99. https://www.buildblockchain.tech/newsletter/issues/no-99-validium-and-the-layer-2-two-by-two

13. Vitalik Buterin (2019). Sidechains vs Plasma vs Sharding. https://vitalik.ca/general/2019/06/12/plasma_vs_sharding.html

14. Vitalik Buterin (2019). The Dawn of Hybrid Layer 2 Protocols. https://vitalik.ca/general/2019/08/28/hybrid_layer_2.html

15. Drew Stone (2018). An overview of PHANTOM. https://medium.com/@drstone/an-overview-of-phantom-a-blockdag-consensus-protocol-part-3-f28fa5d76ef7

16. Drew Stone (2018). An overview of SPECTRE. https://medium.com/@drstone/an-overview-of-spectre-a-blockdag-consensus-protocol-part-2-36d3d2bd33fc

17. Vitalik Buterin (2017). Sharding FAQ. https://vitalik.ca/general/2017/12/31/sharding_faq.html

18. Jeff Coleman (2015). State Channels. https://www.jeffcoleman.ca/state-channels/

documentation

19. Ethereum: https://ethereum.org/en/developers/docs/scaling/

20. PBFT basic process: https://fisco-bcos-documentation.readthedocs.io/zh_CN/latest/docs/design/consensus/pbft.html

21. StateChannels docs: https://docs.statechannels.org/

videos

22. Finematics (2020). ETHEREUM 2.0 - A GAME CHANGER?. https://youtu.be/ctzGr58_jeI

23. 99 Bitcoins (2019). What is Segwit? Segregated Witness Attacks Simply. https://youtu.be/f3CFUbeehc8

Article: @Buidler DAO

Design: @Coucou

Typography: @Coucou