← Back to Research
Research · August 13, 2026

A routing fault, 30 validators, and the 22% question

Simon Morley·14 min read

On the morning of 12 August 2026, a malformed route originating from TeraSwitch's Miami facility propagated through a route reflector in Amsterdam and into European and Asia-Pacific markets, where local routers preferred it over the valid path. Twelve sites lost reachability. Solana kept producing blocks throughout, and no penalty was applied to anyone's principal, because Solana does not slash. What operators lost was block rewards and vote credits.

27
validators simultaneously out of consensus at our observed peak
50.9M
SOL that stopped voting at peak, about $3.9B
18 min
from our first observation to full recovery
22.21%
of all Solana stake sits on one provider's address space

Public reporting, drawing on analysis from Marinade, put the peak at 28.83% of staked SOL out of consensus, within 4.51 points of the 33.34% threshold at which transaction finality stalls. This piece is not a retelling of that. It is what our own collector recorded, minute by minute, and what fell out of the address data once we went looking for why these particular validators went dark together.

What our collector recorded

Observation window 04:18 to 04:38 UTC.

Our Solana collector polls the validator set every 60 seconds and writes a row per validator per state transition. The table below is the raw shape of the event, filtered to the delinquency transitions and their recorded recoveries.

Two facts about this record need stating up front, because they bound everything after. First, our first clean observation of the incident is 04:19:17 UTC, by which point recovery was already underway upstream. Second, the twenty-nine polls before that returned nothing usable. The section on coverage deals with that gap directly.

Poll (UTC)ResultNew transitionsStill outNon-voting stakeNote
03:49:17No data0n/an/aRPC endpoint begins returning HTTP 522
03:50 to 04:17No data0n/an/a28 further polls, same failure
04:18:17OK00n/aFirst clean poll
04:19:17Burst272750.9M SOL27 validators land in one tick
04:20:17Burst22340.6M SOLRecovery outpaces new failures
04:21:17Burst12037.7M SOL
04:24:17Recovering01417.9M SOLHalf the peak cleared in four minutes
04:26:17Burst11113.3M SOLLast new transition of the event
04:32:17Recovering053.5M SOL
04:37:17Recovering010.3M SOLFinal validator recovers
04:38:17Clear00n/aEvent closed

Validators out of consensus, by poll

0714212827 validators50.9M SOL not voting04:1804:2204:2604:3004:3404:38UTC

Marked points are polls that recorded a transition; the line interpolates between them. The curve descends from its first observed value because we arrived after the upstream fault had begun clearing, so this is the tail of the event rather than its shape.

Attributing it, without taking anyone's word for it

Resolving 30 addresses against the regional registries.

A correlated failure is only interesting if you can say what correlated it. We hold a node address for most validators we track, so rather than reading it off a status page we resolved every affected validator's advertised address against ARIN's registry and grouped by the owning organisation.

Twenty-seven of the thirty affected validators sat inside address space registered to TeraSwitch Networks Inc., spread across six separate prefixes. That is not a single rack or a single announcement, it is the provider's footprint.

OrganisationPrefixes observedValidators
TeraSwitch Networks Inc.64.130.32.0/20, 64.130.48.0/20, 63.254.160.0/19, 198.13.128.0/20, 202.8.8.0/22, 67.209.52.0/2227
Helius207.241.184.0/241
Latitude.sh103.106.58.0/231
RIPE region, unresolved to operator84.32.176.0/241

Three of the affected addresses, 63.254.165.79, 63.254.165.80 and 63.254.165.81, are consecutive. Those are three validators, run by parties who in all likelihood believe they are independently hosted, sharing what is almost certainly one rack and one top-of-rack switch.

The largest single casualty by stake was not on TeraSwitch. Helius, on its own address space, had 15.98M SOL stop voting, roughly a third of the total non-voting stake we recorded. A leaked route degrades transit for networks that merely traverse it, not only for the leaking provider's direct customers.

The finding that outlives the incident

Concentration, measured across the whole tracked set.

Having established the prefixes, the obvious next question is not how many validators went down, but how many could have. We ran the same six prefixes against our entire tracked Solana validator set.

Ninety-three of the 829 Solana validators we track sit on TeraSwitch address space. Between them they hold 97.64M SOL, or 22.21% of stake.

Share of Solana stake, against the finality threshold

33.34% finality halt11.58%Non-voting at peak, as observed by our collector28.83%Non-voting network-wide, per Marinade analysis22.21%All stake hosted on TeraSwitch address space0%10%20%30%40%

The first two bars measure one morning. The third measures a standing condition. Our own figure is lower than the network-wide one for the reason given under coverage, below.

Only 27 of those 93 validators actually dropped on 12 August, because the leak affected twelve sites rather than the provider's whole footprint. The three largest TeraSwitch-hosted validators we track, holding 8.30M, 7.97M and 6.12M SOL, stayed up throughout.

So the widely reported near-miss was a partial failure at a provider whose complete failure would remove 22.21% of stake on its own. That is two thirds of the way to a finality stall from one organisation's routing table, before anything else goes wrong anywhere else on the network.

This figure is a floor, not a ceiling. We hold no resolved address for 168 of the validators we track, 20.3% of the set, holding 61.77M SOL between them. Every one of those we later attribute can only add to some provider's concentration; none can subtract from it.

How the recovery actually went

Distribution across 31 recorded transitions.

Recovery was fast and very unevenly distributed. The median validator was back in consensus three minutes after we first saw it drop. The slowest took seventeen.

StatisticMinutesReading
Fastest1Recovered on the next poll
Median3Half the set back within three polls
90th percentile13A long tail carried most of the duration
Slowest17Last validator clear at 04:37:17

The shape matters more than the averages. A routing fault resolves for everyone at roughly the same moment, so a tight cluster of recoveries would be the expected signature. Instead the tail runs four times longer than the median, which is the signature of hosts that needed to catch up, restart, or be intervened on after connectivity returned. Recovering the path is not the same as recovering the validator.

For scale: across the preceding thirty days, Solana averaged 2.3 delinquency transitions per hour. The 04:00 hour recorded 31. That is roughly thirteen times the running rate, and it is why a correlated-failure detector fires on this and not on ordinary daily churn.

What we could not see

A 29-minute observation gap, and what it costs the numbers above.

Our own numbers are lower than the network-wide reporting, and the reason is not a modelling difference. It is that we were not watching for the first half hour.

From 03:49:17 to 04:17:17 UTC, twenty-nine consecutive polls of our Solana RPC endpoint returned HTTP 522. We collected nothing in that window. Our first clean poll was 04:18:17, and the burst of 27 validators that landed at 04:19:17 was not the event beginning, it was us finally seeing an event already in progress and already clearing.

The cause is worth stating plainly, because it generalises. A public RPC endpoint reaches us across the same internet the route leak was busy breaking. The dependency we use to observe the network failed because of the thing we were trying to observe. Across the trailing thirty days our Solana collector recorded 34 failed polls in total, and 29 of them fall inside these two hours, against a baseline failure rate of 0.1%.

So every figure in the collector table should be read as a floor on the true peak. We measured 11.58% of tracked stake out of consensus at our observed maximum; Marinade, watching continuously, measured 28.83% network-wide. We do not think those two numbers are in conflict. Ours is the tail of the curve, sampled after recovery had started.

The attribution and concentration work is unaffected by any of this. Address ownership and stake concentration are standing properties of the network, not measurements of a moment, and they do not depend on having watched the incident live.

What we take from it

Reading the event as a standing condition rather than a morning.

The honest summary of 12 August is that nothing broke. Blocks were produced, transactions landed, finality held with a few points to spare, and the operators involved lost some rewards. Solana does not slash, so no delegator's principal was ever exposed. Treated as an incident, it closed itself inside half an hour.

Treated as a measurement, it is more uncomfortable. The event revealed that a fifth of Solana's stake answers to one organisation's routing decisions, and it did so by exercising only a fraction of that exposure. The 33.34% threshold is not approached by a hundred independent operators each having a bad day. It is approached the way it was approached on 12 August: by many validators discovering at the same instant that their independence was a billing arrangement rather than a physical fact.

Three things follow for anyone holding or running stake:

  • Provider is a risk dimension, and it is usually invisible. A delegator comparing validators sees commission, uptime and self-stake. None of those reveal that two of the candidates sit in the same building on consecutive addresses.
  • Diversification has to be measured at the address, not the brand. Distinct operator names, distinct websites and distinct monikers say nothing about the physical topology underneath. We only found this by resolving addresses to registries.
  • Correlation is the whole risk. Thirty validators failing independently is a statistical non-event. Thirty failing together, in one minute, is 12% of stake leaving consensus at once, and it got there through a mechanism nobody staking with those validators had visibility into.

We are publishing the address-level concentration work because we think it is the part that keeps mattering after the status page goes green.

Method and caveats

How to check us.

  • Source. Every timestamp, transition and stake figure above comes from our own Solana collector record, polling the validator set at 60-second intervals, plus the per-poll outcome log used for the coverage section.
  • Attribution. Node addresses come from validator gossip as we observe it, resolved against ARIN RDAP on 13 August 2026 and grouped by the registered organisation. One address falls in RIPE space and we have not resolved it to an operator.
  • Stake figures. Stake is as recorded at the time of writing, against 439.7M SOL total across 829 tracked validators. Solana stake moves slowly, so a one-day lag between the event and the concentration snapshot is immaterial at this precision.
  • Coverage. 168 tracked validators, 20.3% of the set and 61.77M SOL, have no resolved address. All concentration figures are lower bounds.
  • Duration. All durations run to the observed recovery, never to when our sweep noticed. The 18-minute figure spans 04:19:17 to 04:37:17.
  • Third-party figures. The 28.83% network-wide non-voting share, the 33.34% threshold, the twelve affected sites and the 333 SOL of forgone rewards are from public reporting and Marinade's analysis, not our measurement, and are labelled as such wherever they appear.
  • Terminology. Solana has no slashing of delegated principal. Penalties here are missed rewards and vote credits. We use "out of consensus" and "non-voting" rather than any loss-of-principal language deliberately.

Slashr tracks validator incidents and infrastructure risk across Solana, Ethereum, Cosmos, Celestia, Polkadot, Sui, Avalanche and Near. This analysis was assembled from our own collector record; the underlying incident data is on slashr.dev.

solanavalidator-securityinfrastructure-securitybgproute-leakstake-concentrationslashrincident-analysis
Publication policy:Why we publish these findings →

Related Posts

Beyond Zero answers what the machine decides. It does not answer who said it could.

·5 min read

Google's Beyond Zero moves the trust boundary from the application to the individual action and puts a reasoning layer in the access path. The objection that landed on Hacker News, that a non-deterministic evaluator does not belong there, is aimed one layer too low. The gap is not determinism. It is legitimacy: nothing establishes that a given system has the standing to act in your environment, against your traffic, under your conditions. Vendor-asserted accuracy is not evidence.

earned-autonomyzero-trustbeyond-zero+6 more