Solana - When Mainnet?

Primer

By Andrew Allen

Solana is a performant, permissionless blockchain boasting fast transactions and low fees. The team that’s building Solana is very strong and the ecosystem has attracted dedicated investors: FTX / Alameda, Jump, Multicoin, etc. Solana has also drawn in a community of passionate users, builders, and validators, establishing itself as one of the most popular blockchains in 2022.

Solana’s success over the past year hasn’t been without issues. Because it’s a fast blockchain with low fees, the network has suffered from periods of congestion as bots compete for blockspace. At the extreme, the blockchain has even halted under the load. 

Earlier this year, the network struggled under heavy spam. However, Solana core contributors have outlined a clear path towards handling spam going forward. The implementation of these upgrades has already begun. We’ve seen the foundation of these changes reach production, and a full rollout of these upgrades is being targeted for the end of the next major release series, 1.11.

Solana’s planned upgrades are in many ways the most important yet. If Solana is able to continue processing users’ transactions under spam, it could really cement itself as a go-to blockchain for users and developers alike.

Built for speed

Solana started with the goal of building a blockchain at Nasdaq speed. Their original vision was to even launch an on-chain order book that could compete with the likes of stock exchanges (it ended up happening when FTX engineers built Serum). In building for speed, they got rid of the mempool, allowing requests to flow directly to block leaders, built out Proof-of-History to enable asynchronous validator voting, and developed their own runtime that features parallel execution of transactions and separates the fees for execution (TX fees) vs. memory allocation (cost of maintaining state bloat). Their approach focuses on squeezing the most performance out of hefty machines, relying on hardware to continue getting faster over time to meet increased network demand in the future.

Solana, in many ways, is a bet on scaling a blockchain through engineering versus research. Many other blockchains are expecting to scale leveraging technology still being researched (sharding, zero-knowledge tech, etc.) whereas Solana is focused on solving engineering problems at the networking and execution layer as hardware continues to improve over time.

Also, since they were (and still are) optimizing for fast slot times and high throughput, it was decided to keep transaction fees deterministically cheap. If the chain could process all of the incoming transactions quickly enough, why should anyone pay more? Fast and cheap, that’s the dream. 

Solana’s cheap, fixed fees, however, are in stark contrast to the fee mechanism of most other blockchains. Global fee markets have been the standard approach for handling blockspace demand, and in turn, fees for all use cases have risen even if only one application on the chain gains traction. As fees rose on other blockchains, users could still transact on Solana for near zero cost (0.000005 SOL per signature). Solana’s contrarian approach became a selling point early on.

Fast forward

Jumping to the present day, we see a bustling ecosystem that has gained a large amount of traction in a short period. Some high level items to note (*neither metric is sybil proof):

Source: Unique Fee Payers per Chaincrunch Solana Overview as of 9/8/2022

The number of unique fee paying accounts by day has risen to as high as 400k at its peak and has stayed above 200k for most of 2022. This is a rough proxy for daily active self-custodied wallets, and since some protocols use a single fee-payer, this metric undercounts the number of unique wallets transacting each day.

Source: Unique Programs Used per Chaincrunch Solana Overview as of 9/8/2022

There are now over 1,000 unique Solana programs with at least one successful interaction daily.

As previously mentioned, however, this rise in popularity has not been without letdowns. Solana has suffered from repeated periods of instability, congestion, and spam.

Teams looking to build applications on a blockchain can be sensitive to building their applications on a blockchain that could hinder or prevent users from interacting.  And users, understandably, are turned away by the idea that their assets could be stuck if the network were to be congested.  Permissionless blockchains require resilience under all circumstances to meet expectations. Solana has successfully created a bustling ecosystem despite these challenges. Making Solana resilient under all circumstances is a top priority.

Given the meaningful adoption and ongoing performance issues, the Solana blockchain is at a pivotal point in its development. A full roll-out of upgrades is underway.

Switching to QUIC

At a network level, most data sent over internet connections is via standard transport protocols, mainly TCP and UDP. These communication standards allow machines to share data in a uniform manner across different platforms.

At a high level, TCP is used when a client needs a guarantee that data has reached the server or when a server needs more control. TCP achieves this functionality through an extensive handshake that establishes a connection between hosts. This connection provides a controlled, ordered stream of packets that are acknowledged on delivery and retransmitted if there are any issues. 

On the other end of the transportation protocol spectrum, UDP is used when a client wants their data packets to reach the server as quickly as possible. However, with UDP, a client has no assurance that their data will actually reach the server. UDP does not provide any ordering for traffic nor any delivery confirmation. It is a minimalistic communication model that is built to minimize overhead and optimize for speed. However, without a handshake, UDP packets can have spoofed IPs and therefore incoming UDP traffic cannot be throttled.

The Solana network, as it currently stands, implements a custom UDP-based protocol for RPC nodes to send signed transactions to validators, the slot leaders specifically. Since this communication protocol does not require nodes to establish any connection before sending data, bot operators that want their transactions to reach validators first have begun spamming validators with transactions. At peak load, 100+ GB/s of spamed transactions were purportedly hitting some validators. This ingress is enough to overwhelm even the heaviest validator setups. With the current UDP-based protocol, Solana validators have no way of rate limiting incoming spam.

Enter QUIC, a newer transport protocol developed by Google engineers in 2012 as a quicker alternative to the TCP protocol. QUIC leverages a lightweight handshake that only takes place on initial connection to verify and establish control over packet streams. At its core, QUIC streams are built on top of UDP connections, and so after the initial handshake, packets are sent between hosts as quickly as they would be via UDP.

Most importantly, QUIC will allow validators on the Solana network to rate limit incoming transactions, closing off any streams that are spamming transactions.

Source: TCP vs. QUIC Handshake (modified from source)

With the majority of stake now running 1.10.32+ on Mainnet Beta, QUIC ports are set as the default for validators. The Solana team is now urging RPC providers to switch from UDP to QUIC. Once fully rolled out, validators’ UDP ports can be closed and completely replaced with a QUIC port.

Stake-weighted Quality of Service (QoS)

Slot leaders have limited bandwidth for receiving incoming packets. As such, once QUIC is fully implemented, the team has proposed to stake-weight the incoming traffic being propagated to slot leaders. Instead of accepting incoming traffic on a first come, first serve basis, slot leaders will now process transactions propagated from other validators based on their stake-weight first.

This change allows stake to effectively control the networking stack of the Solana blockchain. Said differently, a Solana validator can leverage their stake, however small, to influence which pending transactions reach other validators even when they aren’t the slot leader. Mechanically, RPCs and validators will have each established a connection with the slot leader beforehand, and depending on the amount of incoming traffic, the slot leader will be able to prioritize servicing transactions propagated by nodes proportional to their stake. And crucially, if there’s no bandwidth available for bots who spam transactions from nodes with no stake, their pending transactions won’t be processed.

For example, a validator with .1% of staked tokens will reserve the right to transmit at least .1% of the incoming packets to the slot leader. No bot operators or other validators will be able to drown out this .1% staked validator’s incoming traffic by flooding the slot leader with their own packets. If there is congestion and not all incoming packets can be handled, the bot operators and other validators respective QUIC streams will be rate limited to match the percentage of stake they each have on the network.

This unlocks a great deal of latent value for validators on the network. In a PoS network, where stake gives a validator the opportunity to propose a certain percentage of blocks, there is usually no way for validators to influence the network outside of selecting the transactions included in their slot and signing other valid blocks. With this change, by stake-weighting incoming packets, validators will now control the odds of pending transactions reaching a leader and being processed quickly.

Validators will each be responsible for allocating their stake-weighted bandwidth on their own. RPC providers with no stake themselves will likely no longer send their transactions directly to the leader, but instead send transactions to other validators on the network first for them to be forwarded to the leader. Validators might be able to price their bandwidth and strike deals with network participants for their advantageous position. No matter how small the amount staked is, a validator can guarantee their allotment of transactions will reach the leader even when the network is under load.

Once fully rolled out, pending transactions forwarded by other validators to the current slot leader are more likely to be included in blocks than any other incoming transactions, especially during times of network congestion.

Local fee markets

Since launch, transaction fees on Solana have been fixed and low (0.000005 SOL per signature). This is changing. 

An additional prioritization fee has been implemented, and we already see 6% of non-voting transactions have an additional fee on top of the base. As it is with the base fee, 50% of the prioritization fee will be sent to the leader for including the transaction and 50% of the prioritization fee will be burned.

Source: Percentage of Solana transactions (non-voting) that include an additional fee

Users are now able to have their transaction included over others by adding this prioritization fee. However, as it currently stands, there’s no well-defined market where users can get an accurate estimate of what fee is necessary to get their transaction included. It’s likely that most priority fee usage today is bots competing in this new dimension.

With the lack of an enforced market, there’s a risk that fees will rise globally for all Solana users. Specifically, if users don’t have an accurate estimate of what additional fee is necessary, chaotic pricing could arise in periods of congestion. However, there are two primary factors that mitigate this risk:

• Parallel runtime: Transactions that hit different parts of the state should be processed in parallel, and in turn, be priced separately. Said differently, prioritization fees should only impact the priority of transactions trying to read or write the same state. Then, all other transactions not hitting that state should be processed as normal without a priority fee.

• High throughput: Solana is built to process many transactions per second. As such, a global transaction fee should only arise if there’s more users wanting their transactions processed than there is blockspace for the transactions to be included each second.

In the near term, APIs have been built to help users estimate the minimum priority fee needed in order to get their transaction included. It’s hard to know whether this is enough or if parallel auctions will need to be enforced further. Regardless, there will be an ongoing effort to tune this system, ensuring that the fee markets remain localized vs. global.

A departure from the fixed fee structure is a big change for Solana. The goal is to build a local fee market where hotspots are priced appropriately so users and bots compete on fee instead of spamming their TXs. Ideally, in its final form, parallel auctions could be built into the validator client, taxing spam appropriately and providing users further clarity on what fee will likely get their transactions included.

Solana in the fast lane

The Solana entrance has caused quite the stir. Detractors argue that since it halts under spam, it isn’t resilient enough to be considered for real world use cases. Solana supporters, on the other hand, praise the protocol for its low fees, and argue that a blockchain with high fees also isn’t usable by the masses.

There’s valid arguments on both sides. More importantly though, Solana is changing in a big way. With 1.10 reaching Mainnet Beta – yes, still beta software, the Solana Labs team has started releasing components of their proposed solution to handle spam. Even more exciting, the Solana Labs team is expecting to do a full roll-out of the potential solutions during the 1.11 release series. 

Upon full rollout, Solana could conquer spam once and for all. The upgrades might even be enough to move Solana past beta, going full Mainnet.

Disclaimer