Mysticeti – The Consensus Mechanism of Sui

What is Sui?

Sui is a Layer-1 blockchain with smart contract capabilities, similar to other well known blockchains such as Ethereum and Solana. Sui blocks are created  by a set of Delegated Proof-of-Stake authorities known as validators, which perform checks on transactions in the network, incorporate them into blocks and propose their blocks to all other validators. Sui’s consensus protocol, Mysticeti, then puts transactions into an order before executing and finalising them.

Whilst on the surface this may sound similar to the workings of traditional blockchains, Sui attempts to uniquely optimise for a high speed, and high volume of transactions. Optimisation techniques include eschewing the account based data model employed by many traditional blockchains. In Ethereum, for example, transactions are instructions that detail changes to account balances. Information about an ERC20 token is stored within the smart contract which created it, meaning that all transactions that involve that token must interact with the same smart contract. In this approach, strict total ordering of transactions is needed so as to ensure account balances are accurately adjusted across the network and avoid double spending.

Sui instead utilises an object centric data model. In this model, everything in the network (such as tokens or NFTs) is represented as an object with a clearly specified owner. This data structure enhances scalability because Sui is able to treat transactions differently depending on the type of object they involve. Transactions involving objects that are owned by a single user can be processed in parallel to each other. This is because the owner is already specified within the object itself, and changes to one object do not affect others. Transactions involving shared objects which do not have a specified owner and can be interacted with by multiple users require a stronger consensus process and thus take slightly longer to finalise. Separating transactions in this manner creates “a multi-lane” system to validate transactions.

Mysticeti is the consensus protocol designed specifically for Sui. The protocol’s main aim is to reduce the number of messages that need to be exchanged between validators to reach consensus and execute transactions in as short a time as possible. Currently, Mysticeti can process simple asset transfer transactions (examples of which we will give in section: Two Types of Transaction) in 390 milliseconds, boasting one of the fastest speeds amongst any blockchains currently. This is essential for the Sui platform to deliver scalable decentralised applications for mainstream adoption.

Assumptions in the SUI Protocol

SUI’s protocol requires a relatively strict set of assumptions about its validators based on the byzantine fault tolerance assumption. The proof for this can be read here. This system uses the assumption that in a system of 3f+1 validators, a maximum of f (approximately ⅓) of the validators are byzantine (may deviate from the protocol). The remainder of the validators, 2f+1 (approximately ⅔), are assumed to be honest and adherent to the rules of the protocol. For example in a system with 1 byzantine validator (f=1), a minimum of 4 validators (3f+1=4) would be needed to operate this system. Secondly, SUI’s protocol operates under the assumption that all messages amongst honest validators eventually arrive in a finite time and a receiver can verify the sender.

Under both of these assumptions, the protocol can be proven to be safe and guarantee liveness – the idea that the network can continue validating transactions and blocks even if up to one-third of validators are malicious or offline. Under these assumptions, SUI’s system, including the consensus mechanism Mysticeti, can be described as a Byzantine fault tolerant system. Whilst these assumptions form a realistic framework to operate on, they are stronger assumptions than those needed to prove the security properties of other blockchains. Notably, Bitcoin’s Proof-of-Work consensus mechanism can be shown to be resistant to up to 50% of malicious hashrate on the network.

The Transaction Lifecycle

Two Types of Transaction

To understand how the consensus mechanism Mysticeti fits into the Sui ecosystem, we must first understand the transaction lifecycle. SUI distinguishes between 2 types of transactions, single owned object transactions and shared object transactions. Differentiating between the two types of transactions is a crucial step in SUI’s protocol. As an example, single owned objects transactions would include transfers to and from wallets of assets owned by a single address. Shared objects, on the other hand, can be accessed and modified by multiple parties.

An example would be trading on a decentralised exchange (DEX), where multiple users can simultaneously interact with the exchange at once. Market prices on the exchange are calculated using the buy and sell price dynamics of executed orders and given the first in first out mechanism employed by most exchanges, these transactions would need to be sequenced before execution. Hence, any transaction engaging with a DEX would be counted as a shared object transaction and be validated through the standard consensus path.

These transactions are broadly processed in the same way – this is with the exception of the actions needed to find consensus, where the path to inclusion on the blockchain is different for both types. There is an accelerated fast path (Mysticeti-FPC) and a normal path (Mysticeti-C) for both transaction types respectively.

First Step – Signing a Transaction

The initial stage of the transaction’s lifecycle is the transaction request - the user will submit this and sign it using their private key. As an example, suppose user A is sending an NFT she owns to User B. Once User A signs her transaction, her wallet application is responsible for the rest of the process, including submitting her transaction to the validators and communicating the outcome of the transaction. There are many validators in the system at any one time which play a part in validating and processing the transactions and these are referred to as a “committee of validators”. The committee is defined as the number of validators chosen for a set time period: in Sui, this time period is set to 24 hours and is referred to as an epoch.

Second Step – Transaction Certificate

Each validator in the epoch then performs a series of validity checks on the transaction. Pre-validation steps include checking that the signature of the user is valid, whether the object actually exists, if the user requesting the transaction truly owns the object, and if there is an adequate amount of gas fees provided to pay for this transaction.

Each validator that agrees that a transaction passes these initial checks must “lock” each object involved in the transaction and return a signature to the user’s wallet application to confirm this. This lock prevents them from being used in other transactions and thus restricts double-spending.

Using the Byzantine fault tolerance assumption, a minimum number of ⅔ of the total validators must provide signatures confirming the lock and only then is a transaction certificate assembled by the user’s wallet. This certificate is returned to the user as a proof that the transaction has been validated by a supermajority of validators. It is confirmation that validators will attempt to execute their transaction but not a record of execution or the transaction being recorded on chain.

In the protocol, the transaction certificate means any other validator (including those that voted on neither transaction) can safely assume the lock and reject any future transactions that attempt to interact with the same objects until the locked transaction has been executed. Having an early fail safe procedure ensures invalid transactions or double spending attempts are prevented early on in the life cycle without requiring the expense of consensus.

Third Step – The Consensus Step

With this transaction certificate, we can move to the third phase. This involves the wallet application sending the transaction certificate back to the validators to check its validity- this means they check whether it has indeed been signed by a majority of validators.

At this point, transactions are sent to the consensus mechanism - Mysticeti. This is where the transaction’s path differs between object types. Single owned object transactions can reach the execution stage using a fast path referred to as Mysticeti-FPC and shared object transactions always go through the normal path, Mysticeti-C. This is because shared object transactions need to be ordered with respect to each other given their interdependent nature.

Fourth Step – Execution

Reaching consensus is the confirmation that the order will be executed and added to the chain. Execution happens almost instantaneously after consensus. Following this, validators will independently sign the effects of the transaction which forms an effects certificate. This is proof of settlement – that the transaction was carried out, added to the blockchain and finalised by all network participants.

Mysticeti - The Consensus Engine

Mysticeti-C

Mysticeti-C, SUI’s consensus mechanism, uses a system of block proposals, rounds, and leader blocks to reach consensus (an agreement) on which transactions can be executed and in which order. The blockchain progresses through rounds where each validator proposes a new block in each round referencing blocks from earlier rounds. Once a block is completed, the validator will add its signature and it can be shared to the network – the next round is initiated.

A consensus cycle consists of 3 rounds: validators propose a block in round 1 containing new transactions, which is “supported” (referenced) by the block proposed in round 2 and “committed” (referenced) by the block proposed in round 3. For a more efficient consensus mechanism, there are multiple consensus cycles, at different stages (propose, support or commit) running simultaneously. For example, as seen in figure 2, in round 3 Validator 1 has proposed a new block and by referencing the blocks from round 2. This block implicitly “supports” the blocks proposed in round 2 by referencing them and “commits” the blocks proposed in round 1 (which was supported in round 2) by referencing the blocks in round 2 that supported it.

The goal of validators in this process is to attempt to come to consensus on whether to accept a block of transactions onto the block chain (commit) or discard the block and unlock the transactions within it (skip). The protocol is expected to either commit or skip the majority of blocks within a 3-round consensus cycle. However, there will be cases where a decision is not made within 3 rounds and the blocks remain in an “undecided” state where the decision to commit or skip is postponed until a later round. The full path to a consensus decision on each block in the Mysticeti consensus protocol is determined by 4 rules:

• direct commit
• direct skip
• indirect commit
• indirect skip

Committing refers to consensus being achieved and transactions being executed. Skipping means the opposite – transactions are rejected and will need to be resubmitted. The direct commit rule is the optimal way of reaching consensus, and this route totals three message rounds while indirect rules can take a minimum of six rounds for the decision.

However, these indirect rules are put in as a fail-safe feature so that in outlier cases, such as a temporary outage of many validators, the blockchain can still progress and the remaining validators can continue to propose blocks. While validators perform all 3 actions – propose, support, and certify blocks – implicitly each round, we have separated these into three distinct phases for simplicity in explaining the process.

Direct Commit Rule

Using the byzantine fault tolerance principles laid out in the assumptions section, in a system of 3f+1 validators, all validators propose a new block in each round, each with a link to at least 2f+1 blocks from the previous round. Round 1 is the Propose phase, where each validator independently proposes a new block containing transactions submitted for approval by other validators. Round 2 is the Support phase, where validators each propose new blocks that “link to” at least 2f+1 blocks created in round 1. When 2f+1 or more blocks in round 2 link to a block from round 1, the round 1 block is “supported”. By linking their round 2 block to a round 1 block, the validator expresses a vote in favour of committing the round 1 block.

Certification in round 3 follows the same principle: if a minimum of 2f+1 blocks proposed in round 2 that support the block in round 1 are themselves supported by 2f+1 or more blocks in round 3, the round 1 block is said to be certified. At this point, proposed blocks from round 1 which have achieved support in round 2 and certification in round 3 can be committed for execution.

This pattern constitutes the direct commit rule. During a round, each validator will have its own view of the system based on the blocks it received from the previous rounds and will independently collate these to create a new block. At the end of the round, the single validator will sign their proposed block and release it to the network. If the committee of validators has a concurring view on the state of the network and blocks within it, consensus is reached.

Direct Skip Rule

Conversely, if only f or fewer blocks (in a system of 3f+1 validators) in round 2 reference a proposed block from round 1, as seen in figure 4, this would constitute a direct skip. Since the proposed block did not gain enough support, it is effectively ignored in the consensus process and transactions contained within this block would have to be resubmitted in a future block. Validators would see this and know that the lock assigned to the objects transacted in this block can be unlocked for subsequent attempts at inclusion in a transaction.

The Indirect Rules

If a block doesn’t get the 2f+1 block links needed in round 2 to be supported, but receives more than f links and so cannot be directly skipped through the direct rules above, it is in an “undecided” state. This situation can also arise in the support round (2), or in the certify round (3) if it receives between f and 2f+1 certification votes.

When undecided, the block’s final status will be determined in subsequent rounds through indirect referencing – when a later block which directly references this initial block is directly committed or skipped successfully as per the rules laid above. In this case, the approval of the later block is “inherited” by the undecided block from the committed or skipped block via its link to it.

For example, figure 5 shows 2 blocks both in the undecided state. The block proposed in round 1 has received 2 votes out of a possible four – not enough support to directly skip commit and too much support to directly skip. This is the same case with the block proposed in round 2. Both their statuses will be revisited in round 4 when a new block, directly referencing the initial blocks, is proposed. This new block assigned with determining the fate of the undecided blocks is referred to as a “leader block” - we explain how it is chosen in the next section. Therefore a commit, skip or continued undecided fate is assigned in round 6.

The undecided block/s can be referred to block/s A and the new “leader” block proposed in round 4 as block B. In figure 6, a flowchart outlining the 3 outcomes of the indirect rules is shown visually. The final state of A can be determined by whether B is directly committed, directly skipped or also in an undecided state. If B is committed, block A will be committed by the “indirect commit rule”; if B is rejected, A will also be rejected and this is referred to as the “indirect skip rule”. If block B is undecided, A will remain undecided also.

In the rare case where block B remains undecided, this would likely indicate a faulty or malicious validator in the system and block A would be subject to a timeout period. In this case the transactions contained in block A will be returned to the user as unsuccessful and they will get the opportunity to resend their transaction through an alternative validator. Block B, now branded as undecided, would be subject to the same process block A went through above. This scenario could happen for example if the first validator crashed and is therefore temporarily excluded from the network.

Leader blocks

One block in each round is designated a “leader block”, which is necessary for the use cases of the indirect rules described above. Validators are made aware in advance of the ordering of the leader blocks – typically by round-robin (changing per round). This failsafe mechanism ensures liveness and caps transaction times so undecided blocks’ fates can be decided within the next consensus cycle (as opposed to blocks remaining undecided indefinitely which could potentially pause the blockchain progression as undecided blocks accumulate).

Each round has a primary leader (Leader 1a) and alternative leaders (Leader 1b/1c…) which serve as a back up in the case that a leader block is not proposed by the assigned validator, or the leader block is not received within the time limit. The validator in charge of proposing the leader block is predetermined and randomised at each round, removing the opportunity for malicious validators to repeatedly delay the chain's progression by delaying block distribution in consecutive rounds.

Since the indirect rules outcome is determined by one leader block, it removes the latency associated with checking every block. This mechanism works under the assumptions described above that 2f+1 out of the total validators (3f+1) are behaving honestly and that messages between validators will be received in a finite time. With this system, the blockchain is able to continue in the presence of delays or faulty nodes.

Mysticeti-FPC – Fast Path Consensus

Mysticeti-FPC is an alternative route to Mysticeti-C, designed to provide even quicker execution times via the fast-path. It follows the same core mechanisms but with a key difference: execution can occur at the end of round 2. This accelerated fast path execution is only suitable for single owned object transactions, where all the inputs are owned objects. Transactions involving shared objects do not qualify for this fast path and are processed using the Mysticeti-C consensus mechanism described in the previous section.

When validators are composing their blocks, they can include a mix of both types of transactions. They already know from their pre-validation checks which transactions only involve single owned objects and which involve shared owned objects. In general, when validators include transactions in a block, they are implicitly voting on the validity of that transaction. So far, this follows the same process as in Mysticeti-C.

Blocks proposed in the first round are followed by a normal supporting round, where 2f+1 blocks in round 2 must reference the initial block that contains the fast path transaction for it to gain sufficient support. The difference between the two paths occurs here – once this threshold is reached, and the block containing the fast path transaction has received 2f+1 support, it is immediately executed at the end of round 2. In the standard path, transactions must reach round 3 for certification before execution, making the process more efficient without delaying the transaction's finalisation.

Execution and Finality

Mysticeti-FPC makes a distinction between fast-path execution and fast-path finality. Execution is when a single validator has locally executed the transaction, and the object is now unlocked and can now be used in subsequent transactions. As shown in the figure above, fast-path execution is completed immediately after the pattern of 2f+1 votes is identified – each of the validators that has supported the block executes the transaction on their local machine. Finality, shown in the figure below, follows round 3, the commit round, and confirms that the transaction is final across all validators and each node on the network knows that the effects of the transaction are unable to be reversed. This is similar to Mysticeti-C, where 2f+1 certifying blocks link to 2f+1 supporting blocks as shown in figure 8.

Alternatively, a fast path transaction can also be finalised when the proposed block containing the transaction has a certify block which includes 2f+1 votes for the transaction in its causal history but has also already been committed by Mysticeti-C. Figure 9 below shows this pattern – the green block in round 3 is a committed block from Mysticeti-C and can finalise fast path transactions included in the first block in round 1.

The Delegated Proof of Stake Mechanism

The Sui platform relies on delegated proof of stake to determine the validators every 24 hour period (epoch). Delegation is implemented by allowing any user of the platform who holds SUI tokens to delegate some or all of their holdings to a specific validator and earn a yield on their stake. These tokens are then locked for the whole epoch. Users are able to choose which validator they want to delegate tokens with, often via their wallet application which gives a list of validators in the epoch (we explain the requirements needed to become a validator below). This means some validators may be delegated more SUI tokens than others. All validators however, have an equal role in transaction validation and execution.

Users are charged a gas fee whenever they make a transaction on the network. Staking rewards are generated from the gas fees collected throughout the epoch as well as stake subsidies released at the end of each epoch (24 hours). Stake subsidies amount to 10% of the total SUI token supply and will be released slowly over time until April 2030, according to a predefined emission schedule by the Sui network. When the subsidies finish, stake rewards will be entirely made of gas fees. Since processing a transaction requires participation from various validators, all honest validators can receive the rewards generated from gas fees, in proportion to their share of delegated stake.

Before every epoch begins, validators are asked to submit a reservation gas price. This is the minimum gas price at which they are willing to process transactions. The Sui protocol then collates the prices submitted, and chooses a value such that 2f+1 (a majority) of the validators are happy to process transactions in that epoch. This price is called the reference gas price and is fixed for the duration of that epoch.

The gas pricing mechanism is designed to boost the rewards of validators who processed all transactions above their self-declared reservation price and equally penalise the rewards of validators who did not honour their reference gas price. This intuitively means validators are incentivised to set gas prices low but not too low – otherwise they get penalised for not meeting such prices.

In this proof of stake mechanism, users who hold SUI tokens and delegate them to validators also receive staking rewards, not just the validators. When users choose a validator to stake with, they are made aware of the validator’s commission rate, i.e., how much of the user’s stake reward, the validator will take. SUI token delegators receive the same boosted or penalised rewards as the validator they delegated their tokens to. Any validator who is unresponsive or fails to validate transactions will have their rewards slashed directly, but will also likely have a smaller delegated stake in future epochs as users move their tokens with more responsive validators. This means their rewards subsequently get smaller in a twofold manner.

At the time of writing (Nov, 2024), there is a total of US$26 billion staked by validators and delegated to validators in the Sui network, across 107 validators. In order to become a validator on the Sui network, a total of 30 million SUI tokens are required, which at a current price of $3.4 (Nov 26, 2024), is equal to US$102 million. If a validator’s stake falls below 20 million SUI, they are given a grace period of seven epochs to gain back the stake, otherwise they are removed from the validator set.

This high barrier to entry means becoming a validator can be exclusionary to the wider crypto community. However, since validators and their delegators get boosted or penalised rewards based on validator behaviour and the proportion of stake they have, there is an incentive for good behaviour.. This system encourages users to delegate their stake, rather than become validators themselves – in this way, regular SUI token holders are more involved in the staking mechanism and willing to switch to other validators should any misbehave. All else held equal, fewer validators means less communication and message delays, which plays a role in supporting faster transaction processing times.

Delegation of tokens and less validators does have a trade-off – we know if a bad actor controls f+1 out of 3f+1 of the total staked SUI, they can attack the network. This introduces a risk of centralising the voting power and threatens the assumption of 2f+1 honest validators. As of November 2024, the total staked SUI stands at US$26 billion. This implies that the system's security assumption, that less than f of the validators are malicious, remains valid as long as the maliciously controlled stake does not exceed US$8.6 billion.

Conclusion

Sui is able to achieve impressive transaction speeds and scalability due to its object-centric data model and its latest upgrade to the new Mysticeti consensus protocol. This mechanism enables efficient parallel processing of single-owner object transactions while processing interdependent and shared-object transactions through a streamlined consensus process.

Improvements to the consensus mechanism come with a plethora of benefits for both the users of the Sui platform and its validators. These include: greater resource efficiency for validators  (as less computing power is required), faster transaction times, and a greater throughput of transactions processed per second. Hence, Mysticeti enables faster parallel execution of transactions and reduces the bottleneck of traditional sequential blockchain consensus protocols. Some of the trade-offs required to achieve this include the stronger trust assumptions of 2f+1 honest validators (compared to other consensus algorithms), and the potential risk of centralisation, given the costly barrier to entry.

The SUI blockchain has a range of uses including high-frequency, real-time applications like gaming which require minimal transaction lag, alongside everyday wallet transactions. SUI’s consensus mechanism: Mysticeti-C, uses structured rounds, leader blocks, and defined commit and skip rules to optimise latency and throughput, while establishing security through the pre-defined assumptions. It also has rules in place to ensure blockchain progression even in the presence of faulty or malicious validators.