Solana (SOL) narrowly misses network shutdown after major data center failure

Core Points

Infrastructure centralization becomes major vulnerability

On Wednesday, about 29% of pledged tokens in the Solana network were offline due to a routing failure in the Teraswitch data center. Solana was once only about 20 million SOLs short of the key threshold for triggering network paralysis (33.34% offline rate). The infrastructure failure began at Teraswitch's hub in Miami and later spread to multiple facilities in Europe and Asia. The proportion of pledged SOL held by a single network operator exceeds 25%, which exceeds the security parameters recommended by Solana. According to Solana Foundation officials, transaction processing and block production were not interrupted throughout the crisis.

Solana network nearly crashed, infrastructure centralization raises concerns

On Wednesday, the Solana blockchain nearly suffered a catastrophic network outage due to infrastructure problems in the Teraswitch data center that left nearly 29% of validators offline. This danger has intensified scrutiny of the network's reliance on centralized infrastructure providers.

The chain reaction of failures originated from Teraswitch's data center in Miami. The technical team discovered that an erroneous default route advertisement spread through a route reflector in Amsterdam, eventually disrupting network connections at facilities in Europe and Asia. Complete network path failures occurred in more than a dozen data center locations, including London, Amsterdam, Dublin, Frankfurt, Singapore and Tokyo, while North American facilities remained operating normally.

Pledge service provider Marinade records show that about 90 verifiers committed dereliction of duty during the outage. Together, these validators control 28.83% of all pledged SOL tokens. Blockchain was dangerously close to a critical point-just about 20 million SOLs short of the key threshold of 33.34% that triggers a complete network failure.

In blockchain systems, finality represents the stage at which a transaction becomes permanent and irreversible. When more than one-third of the pledged tokens are disconnected at the same time, the network loses the ability to finalize transactions, effectively stopping all participant operations in the entire system.

Teraswitch engineers diagnosed the root cause in approximately 10 minutes. Full connection restoration completed at 4:16 a.m. UTC time. The 90 affected validators lost a total of 333 SOL pledge rewards, but Marinade confirmed that these losses will be compensated through the validator bond mechanism.

Infrastructure centralization becomes the main weak link

In addition to direct technical failures, Marinade emphasized that pledge centralization is the fundamental problem. Analysis showed that a single network operator (tracked through ASN AS20326) controlled 27.34% of the total pledged SOL during the peak event period. This concentration exceeds Solana's self-set 25% distribution guideline. About 94% of SOLs associated with this carrier were offline during the outage.

Another 14.1 million SOL tokens were plunged into darkness due to validators hosted through Latitude.sh, Limestone, Butterfly Research and Allnodes. Marinade admitted that it was uncertain whether these secondary interruptions originated from the same routing failure.

The platform frankly acknowledged the challenges of its infrastructure centralization, revealing that four autonomous system numbers control two-thirds of its entrusted pledges. "No one should feel at ease about this, including ourselves," the Marinade team admitted in a post-mortem analysis.

Solana Foundation defends network architecture

Jacob Creech, vice president of technology at the Solana Foundation, characterized the incident as a verification of the infrastructure of the network. He emphasized that 597 of the 699 pledge verifiers maintained voting operations throughout the crisis. The affected validators successfully restored functionality within 40 minutes.

"Because Solana verifiers are distributed among independent infrastructure providers, failures from individual providers did not disrupt the network," Creech said on the X platform.

Solana currently protects the total value of US$4.3 billion locked in the DeFi agreement, despite the network's multiple interruptions in its operating history. A network shutdown in February 2024 took about five hours to fully recover. As of press time, Solana was trading at approximately $75.79.