HomeIncident library
Incident library

INCIDENT REPLAY · unknown

Valve / Steam disconnected from upstreams by a BGP configuration error

Steam became unreachable when Valve's network lost its connections to three transit providers at once.

Fits no familyBGP / WAN routingCatalog confidence: low

A reconstruction, not the operator's network. Hammerhead's authors rebuilt the change from the public record as a small network of their own. No configuration here is the operator's. What follows is what the tool reported on that reconstruction; it is not a claim that the tool would have prevented the outage.

What happened

Who: Valve (Steam)

The summary at the top is paraphrased from the sources below. Hammerhead's catalog rates its confidence in this account low.

Sources

What the public record says

  • Steam became unreachable when Valve's network lost its connections to three transit providers at once.
  • Who: Valve (Steam)

What the reconstruction assumed

Decided by Hammerhead's authors to build a network that runs. None of it describes the operator's network.

Everything below is ours.

  • The public record is one sentence: a BGP configuration error dropped three transits at once. No date, no mechanism. We assume the three neighbour statements were removed.
  • Cisco IOS syntax and stand-in address space throughout. A reconstruction, not any operator's configuration.

What Hammerhead said

Each run below is pinned by Hammerhead's replay tests at commit 0ee9732: the verdict and the quoted text are checked against the tool's output, and a run that disagrees fails the test. Reason lines are copied verbatim; a pinned line can be part of a longer reason. Every preflight run uses --max-blast-radius 100, so a change is never counted as caught merely because it touched many devices of a small network.

Hammerhead's own first-versus-now record

First verdict
fail, but only on the generic forwarding-semantics tripwire, which anyone editing BGP waives with --allow-flow-changes. With that waiver: pass.
Verdict now
fail with the waiver: each deleted session is named with what it announced, and AS 32590 would have no session to any network outside the snapshot.

Quoted from docs/replays/families/individual.md. “First” is the tool before the fixes this replay prompted.

Valve's own edge router

now: fail

On the reconstruction, hammerhead preflight on a before and an after snapshot with --allow-flow-changes returned fail, for the reasons below.

Reason text pinned (some reason contains each line)

  • VALVE-EDGE-1 would lose its BGP session to
  • AS 32590 would have no BGP session to any network outside the snapshot, where it has 3 today

Pinned in tests/gauntlet/replays-individual.yaml.

What changed in Hammerhead because of it

From the “What was fixed” table in Hammerhead's replay notes, where each fix landed after a failing test. “Found by” is copied as written; a family name means the batch this incident belongs to.

A deleted session to a network outside the snapshot

Fell between the adjacency check (modelled pairs) and the advertisement check (peers present in both snapshots), leaving only the generic tripwire everyone waives.

Found by: Valve

Limits

  • The configurations are a reconstruction. Where the record is silent, the assumptions above fill the gap; if one is wrong, the replay is wrong in that respect.
  • A verdict describes the reconstructed network's converged state. Timing, traffic load, and anything a device does that no configuration states, such as software reacting to a health check or a firmware bug, are outside it.
  • A pass is reported as a pass. A fail says a gate fired on the reconstruction, nothing more.
  • Why the catalog counts this outage as in scope: A BGP policy or session change that withdraws an enterprise's prefixes from every upstream is a before/after route-advertisement and reachability diff.

Imported from docs/replays/valve-steam/FACTS.md and the replay manifests in the Hammerhead repository at commit 0ee9732.