HomeIncident library
Incident library

INCIDENT REPLAY · 2009-10-10

Wind Telecomunicazioni 85,000-prefix mis-origination (Italy)

At about 07:22 UTC AS9035 originated roughly 85,000 prefixes that were not its own. Only two BGPmon peers, both in Italy, saw the routes, and the event cleared in about two minutes.

Accepting networkHijack originationCatalog confidence: medium

A reconstruction, not the operator's network. Hammerhead's authors rebuilt the change from the public record as a small network of their own. No configuration here is the operator's. What follows is what the tool reported on that reconstruction; it is not a claim that the tool would have prevented the outage.

What happened

Who: Wind Telecomunicazioni (AS9035) caused; Telecom Italia customers among those affected

The summary at the top is paraphrased from the sources below. Hammerhead's catalog rates its confidence in this account medium.

Sources

What the public record says

  • At about 07:22 UTC AS9035 originated roughly 85,000 prefixes that were not its own. Only two BGPmon peers, both in Italy, saw the routes, and the event cleared in about two minutes.
  • Who: Wind Telecomunicazioni (AS9035) caused; Telecom Italia customers among those affected

What the reconstruction assumed

Decided by Hammerhead's authors to build a network that runs. None of it describes the operator's network.

Everything below is ours.

  • The family template (tools/replays/gen_family_audit.py): PEER-1 (AS 64500, a peer that took 85,000 mis-originated prefixes) has a session to WIND-1 (AS 1267, Wind, re-originating a partial table) with a prefix-list that permits any prefix of any length, no prefix limit, an inbound AS-path filter, and a second session with every safeguard, so that each finding is one row.
  • 85,000 prefixes with the origin rewritten. The peer session that took them is modelled with no prefix limit, which is the one safeguard that would have held it to a number.
  • No change is replayed: the announcing network's change is not ours to gate. The audit runs on the accepting router's healthy configuration.
  • Cisco IOS syntax and stand-in address space throughout. A reconstruction, not any operator's configuration.

What Hammerhead said

Each run below is pinned by Hammerhead's replay tests at commit 0ee9732: the verdict and the quoted text are checked against the tool's output, and a run that disagrees fails the test. Reason lines are copied verbatim; a pinned line can be part of a longer reason. Every preflight run uses --max-blast-radius 100, so a change is never counted as caught merely because it touched many devices of a small network.

PEER-1's own router, no change: no prefix limit

now: fail (finding)

An audit of one snapshot, no change: hammerhead nqe-library --mode run, check nqe.bgp.max_prefix_configured. Pinned status: fail, a finding whose evidence contains the text below.

Evidence pinned (the evidence contains each line)

  • 1 row(s)
  • PEER-1
  • 1267

First run, before the fixes this batch prompted

first: fail (finding)

1 row(s) returned (expected 0); first: s.device=PEER-1 s.peer=172.16.0.2 s.remote_as=1267

Pinned in tests/gauntlet/replays-audit.yaml.

What changed in Hammerhead because of it

From the “What was fixed” table in Hammerhead's replay notes, where each fix landed after a failing test. “Found by” is copied as written; a family name means the batch this incident belongs to.

Inbound max prefix length and AS-path filter on BgpSession

prefix_list_max_length and a new ebgp_inbound_as_path_filter audit stop being stubs: a /32 for somebody else's address wins by longest match wherever it is accepted; a prefix filter generated from a forged registry object says nothing about whose path.

Found by: Accepting-network family

Limits

  • The configurations are a reconstruction. Where the record is silent, the assumptions above fill the gap; if one is wrong, the replay is wrong in that respect.
  • A verdict describes the reconstructed network's converged state. Timing, traffic load, and anything a device does that no configuration states, such as software reacting to a health check or a firmware bug, are outside it.
  • A pass is reported as a pass. A fail says a gate fired on the reconstruction, nothing more.
  • Why the catalog counts this outage as partly in scope: The shape (origin rewritten on a partial table) is what a redistribution or aggregate mistake produces and would show in a pre-push originated-prefix diff, but the cause is undocumented.

Imported from docs/replays/wind-2009/FACTS.md and the replay manifests in the Hammerhead repository at commit 0ee9732.