HomeIncident library
Incident library

INCIDENT REPLAY · 2008-02-24

Pakistan Telecom YouTube hijack via PCCW

Ordered by the Pakistani government to block YouTube, Pakistan Telecom announced 208.65.153.0/24, a more-specific of YouTube's 208.65.152.0/22, at 18:47 UTC. Its upstream PCCW propagated the route globally and YouTube was unreachable for much of the internet. YouTube countered with its own /24 at 20:07 and /25s at 20:18, and PCCW withdrew AS17557's routes at 21:01 UTC, about 2 hours 14 minutes after the start.

Hand-built replaysBGP leak / hijackCatalog confidence: high

A reconstruction, not the operator's network. Hammerhead's authors rebuilt the change from the public record as a small network of their own. No configuration here is the operator's. What follows is what the tool reported on that reconstruction; it is not a claim that the tool would have prevented the outage.

What happened

Who: Pakistan Telecom / PTCL (AS17557), propagated by PCCW Global (AS3491), Hong Kong; victim YouTube (AS36561)

The summary at the top is paraphrased from the sources below. Hammerhead's catalog rates its confidence in this account high.

Sources

What the public record says

The parties and the prefixes

  • 1. The victim's normal announcement. RIPE: "Before, during and after Sunday, 24 February 2008: AS36561 (YouTube) announces 208.65.152.0/22. Note that AS36561 also announces other prefixes, but they are not involved in the event."
  • 2. The addresses that mattered. RIPE: "youtube.com has three IP numbers in the DNS: 208.65.153.238, 208.65.153.251 and 208.65.153.253." All three are inside 208.65.153.0/24.
  • 3. The /24 did not exist in the global table beforehand. RIPE: "The prefix 208.65.153.0/24 is not announced on the Internet before the event".
  • 4. The originator and its upstream. RIPE: "Pakistan Telecom (AS17557) started an unauthorised announcement of the prefix 208.65.153.0/24. One of Pakistan Telecom's upstream providers, PCCW Global (AS3491) forwarded this announcement to the rest of the Internet, which resulted in the hijacking of YouTube traffic on a global scale." Note "one of": AS3491 was not AS17557's only upstream.

Why it was announced

  • 5. It was a censorship measure. REN: "Pakistan Telecom, in response to government order to block access to YouTube (see news item) started advertising a route for 208.65.153.0/24 to its provider, PCCW (AS 3491)." RIPE: "Pakistan aimed to block the YouTube website."
  • 6. The route was meant to be a blackhole, and the announcement was the error. REN, the author replying in the comment thread (26 February 2008): "there were two technical errors. First, Pakistan Telecom was advertising a route they had only intended to blackhole. Second, PCCW didn't have prefix filters installed to limit the reach of this advertisement."

Why it worked

  • 7. Longest-prefix match. REN: "this is a more specific route than the ones used by YouTube (208.65.152.0/22), and therefore most routers would choose to send traffic to Pakistan Telecom for this slice of YouTube's network." And: "According to usual BGP route selection process, the /24 was then chosen, effectively completing the hijack."
  • 8. The upstream did not filter. REN: "PCCW (3491) did not validate Pakistan Telecom's (17557) advertisement for 208.65.153.0/24. By accepting this advertisement and readvertising to its peers and providers PCCW was propagating the wrong route."
  • 9. What would have stopped it, per the two sources. REN: "A provider can and should limit the advertisements it will accept from a customer. The mechanics can be arranged manually or can be configured using Routing Policy Specification Language (RPSL)". REN also notes the size of the job: "In the case of Pakistan Telecom, they originate or transit fewer than 1000 prefixes." RIPE: "unauthorised announcements like this can be prevented from spreading throughout the Internet by appropriate routing configuration by operators of Autonomous Systems."
  • 10. Origin certification did not exist yet. RIPE (March 2008): "Currently the RIPE community is discussing the introduction of digital certificates for Internet number resources." REN lists only proposals: "Cryptographic extensions to BGP have been suggested (Pretty Good BGP, Secure Origin BGP and SBGP). These may be too taxing for router CPUs."

Timeline (UTC, 24 February 2008)

  • 11. Start. RIPE: "18:47 (UTC): AS17557 (Pakistan Telecom) starts announcing 208.65.153.0/24. AS3491 (PCCW Global) propagates the announcement. Routers around the world receive the announcement, and YouTube traffic is redirected to Pakistan." REN: "18:47:45 first evidence of hijacked route propagating in Asia, AS path 3491 17557".
  • 12. Speed and reach. REN: "18:49:30 all providers who will carry the hijacked route have it (total 97 ASNs)", and "we estimate that it was seen by a bit more than two-thirds of the Internet." RIPE: "in the two minutes following the first announcement at 18:47 (UTC), the unauthorised route had spread to its largest extent in the RIS routing view."
  • 13. RIPE is explicit about the limit of its own data: "Based on this information, it is not possible to make statements about how many sites had their traffic to YouTube hijacked."
  • 14. First countermeasure. RIPE: "20:07 (UTC): AS36561 (YouTube) starts announcing 208.65.153.0/24. With two identical prefixes in the routing system, BGP policy rules, such as preferring the shortest AS path, determine which route is chosen. This means that AS17557 (Pakistan Telecom) continues to attract some of YouTube's traffic."
  • 15. Second countermeasure. RIPE: "20:18 (UTC): AS36561 (YouTube) starts announcing 208.65.153.128/25 and 208.65.153.0/25. Because of the longest prefix match rule, every router that receives these announcements will send the traffic to YouTube." RIPE also records that the /25s were "much less visible on the Internet than the /24 prefix": in a later RISwhois lookup of YouTube's own announcements, "The /22 is the one that is most widely seen (by 112 RIS peers). The /24 is seen by 105 peers. The /25 announcement, however, only makes it to 21 of the peers."
  • 16. Prepending. RIPE: "20:51 (UTC): All prefix announcements, including the hijacked /24 which was originated by AS17557 (Pakistan Telecom) via AS3491 (PCCW Global), are seen prepended by another 17557." REN: "20:50:59 evidence of attempted prepending, AS path was 3491 17557 17557".
  • 17. End. RIPE: "21:01 (UTC): AS3491 (PCCW Global) withdraws all prefixes originated by AS17557 (Pakistan Telecom), thus stopping the hijack of 208.65.153.0/24. Note that AS17557 was not completely disconnected by AS3491. Prefixes originated by other Pakistani ASs were still announced by AS17557 through AS3491." REN times it at "20:59:39 hijacked prefix is withdrawn by 3491, who disconnect 17557". The two sources disagree on whether AS17557 was disconnected; we reproduce both and resolve neither.
  • 18. Duration. RIPE: "YouTube reacted about 80 minutes after the Pakistan Telecom announcements, and all the major events finished after about two hours." REN: service "was interrupted for about an hour and forty minutes for some lucky customers and, at most, a bit more than two hours. The exact duration of the outage depends on your vantage point on the Internet."

AS paths on the record

  • 19. One vantage point, AS3333 (the RIPE NCC's own AS). RIPE: "Until Sunday, 24 February 2008, 18:47 (UTC), AS3333 (RIPE NCC) had no path toward 208.65.153.0/24"; from 18:47 to 20:52 it "observed 208.65.153.0/24 being announced by AS17557 (Pakistan Telecom) through two distinct paths (3333 6320 3549 3491 17557 and 3333 12859 3491 17557)"; since 20:52 it "has observed 208.65.153.0/24 being announced by AS36561 (YouTube) through the path 3333 3356 3549 36561".
  • 20. It follows from fact 19 that AS3549 appears adjacent to both AS3491 and AS36561 in observed paths. The sources do not name AS3549, AS6320, AS12859 or AS3356; the AS numbers are all the record gives.

What the reconstruction assumed

Decided by Hammerhead's authors to build a network that runs. None of it describes the operator's network.

None of the following is on the public record we read. Each is a choice we made so that the corpus parses, converges and runs in under a second. If one of them is wrong, the replay is wrong in that respect.

  • A1. The router configuration is ours, all of it. Pakistan Telecom's configuration has never been published. Facts 5 and 6 say a route was "intended to blackhole" and was advertised to AS3491. They do not say how. The widely repeated account, which we remember but did NOT re-read in either source, is a static route to a null interface that reached BGP through redistribution. We model exactly that: ip route 208.65.153.0 255.255.255.0 Null0 on a border router that already carries redistribute static under router bgp 17557 and has no outbound prefix-list or route-map toward AS3491. The word "Null0", the phrase "redistribute static" and the absence of an outbound filter are our reconstruction, not quotations.
  • A2. `redistribute static` is already present in `before`. We put it there so that the national aggregate is originated the ordinary way (a Null0 pull-up static, redistributed), which makes the change a single line: the blackhole static. We consider this the most truthful rendering of "advertising a route they had only intended to blackhole" (fact 6): the engineer adds a blackhole, and a pre-existing redistribution statement turns it into an announcement. The alternative the brief allowed, a BGP network statement, is kept as a labelled control (after_isp_only_network_stmt/) because the gate treats the two differently; see RESULT.md.
  • A3. Vendor and syntax. Every device is written in Cisco IOS syntax because that is where this parser classifies 100% of the lines we needed. It says nothing about what any of these operators ran in 2008.
  • A4. Topology. Seven routers: YT-EDGE-1 (AS36561), GBLX-1 (AS3549), PCCW-1 (AS3491), RIPE-1 (AS3333), and PTCL-BR-1, PTCL-CORE-1, PTCL-AGG-1 (AS17557). One router per external AS is ours. AS3549 sits between AS3491 and AS36561 because fact 19 shows it adjacent to both. AS3333 is the third-party vantage point because it is the one the record gives paths for; we attach it directly to AS3549 and AS3491, which compresses out AS6320, AS12859 and AS3356 (fact 19). AS17557's other upstreams (fact 4, "one of") are omitted. The three-router interior of AS17557, its OSPF area 0 and its iBGP full mesh are ours.
  • A5. Addresses. Only 208.65.152.0/22, 208.65.153.0/24 and 208.65.153.238 are real (facts 1 and 2). We give YT-EDGE-1 a connected front-end LAN 208.65.153.0/24 and a loopback 208.65.152.1, both ours. Everything else is documentation or benchmarking space and stands for nobody's real allocation: inter-AS links from 198.51.100.0/24, transit loopbacks from 192.0.2.0/24, AS3333's LAN 203.0.113.0/24, and AS17557's national aggregate 198.18.0.0/15 with its loopbacks, links and subscriber LAN inside it.
  • A6. The ISP-only snapshot. before_isp_only/ and after_isp_only/ are the three AS17557 files copied unchanged. boundary.yaml declares the AS3491 session and lists what that peer advertises in OUR full corpus; it is not a record of what PCCW Global sent Pakistan Telecom. boundary_undeclared.yaml declares nothing, on purpose.
  • A7. The optional host. before_with_host/ and after_with_host/ add YT-WEB-1, an IOS-syntax stand-in for a web server holding 208.65.153.238. A web server has no router configuration and would not be in anyone's snapshot; it exists only to measure how much the verdict depends on a modelled device happening to own an address inside the hijacked prefix. It is not part of the primary corpus.
  • A8. No countermeasures, no timing. Facts 11 to 18 are a two-hour sequence of announcements, counter-announcements, prepending and a withdrawal. The corpus models one step: the state before 18:47 and the converged state after it. The simulator computes converged routing state and makes no claim about propagation time (fact 12).
  • A9. Propagation is total in the corpus. Every AS in the corpus accepts the /24. The record says the real reach was partial (facts 12, 13 and 15).

What Hammerhead said

Each run below is pinned by Hammerhead's replay tests at commit 0ee9732: the verdict and the quoted text are checked against the tool's output, and a run that disagrees fails the test. Reason lines are copied verbatim; a pinned line can be part of a longer reason. Every preflight run uses --max-blast-radius 100, so a change is never counted as caught merely because it touched many devices of a small network.

Hammerhead's own first-versus-now record

First verdict
pass in every framing. No gate looked at what is announced to a peer outside the snapshot.
Verdict now
fail in both framings. The ISP's own routers with the upstream declared: the /24 "is inside 208.65.152.0/22 that AS 3491 advertises to this network". The whole internet in one snapshot: "208.65.153.0/24 on YT-EDGE-1 (208.65.153.1) from 6 devices" is no longer reachable.
Fixed?
Yes. The realistic framing needs --boundary saying what the upstream sends; without it the new announcement is a caveat. The whole-internet framing was pass until the sweep probed connected subnets as well as one address per device.

Quoted from docs/replays/README.md. “First” is the tool before the fixes this replay prompted.

This replay's runs are pinned in Hammerhead's hand-written Rust tests rather than a manifest, so the record above is the only account quoted here.

What changed in Hammerhead because of it

From the “What was fixed” table in Hammerhead's replay notes, where each fix landed after a failing test. “Found by” is copied as written; a family name means the batch this incident belongs to.

Foreign-space announcement

With a declared upstream, announcing a more-specific of space it sends you is a reason.

Found by: YouTube

Announcement changes count as a BGP change

One ip route line under redistribute static changed what was announced while router bgp stayed identical.

Found by: YouTube

The sweep probes every connected subnet

It probed one address per device, a loopback for choice, and the ledger called that exhaustive. A customer /24 dropped from the IGP, filtered on the way in, or pulled away by a more-specific left every loopback answering. Every connected global-table subnet is now a destination from every probed device; lost subnets are reported apart from lost device pairs. 0.17 s on a 320-device fat tree with 576 subnets.

Found by: YouTube, Verizon, Facebook

New origination of address space learned over eBGP

With the peer inside the snapshot nothing is declared, and no gate compared a new origination with what the network hears from its neighbours. A more-specific of somebody else's prefix wins by longest match wherever it is heard and loses no flow in the model. The covering route is looked for across the whole AS, before and after the change; a default route never counts. YouTube's whole-internet verdict now names the cause as well as the damage: "inside 208.65.152.0/22 that it learns from AS 3491, originated by AS 36561".

Found by: Verizon, YouTube

Limits

  • The configurations are a reconstruction. Where the record is silent, the assumptions above fill the gap; if one is wrong, the replay is wrong in that respect.
  • A verdict describes the reconstructed network's converged state. Timing, traffic load, and anything a device does that no configuration states, such as software reacting to a health check or a firmware bug, are outside it.
  • A pass is reported as a pass. A fail says a gate fired on the reconstruction, nothing more.
  • Why the catalog counts this outage as in scope: Simulating the blackhole-route change shows a foreign /24 being exported to transit AS3491, and PCCW's customer session lacked a prefix filter, both of which are config findings.

Not fixed: Hosts the snapshot does not model

The sweep now probes every connected subnet, by the address the router holds on it. A prefix that exists only as a static or an aggregate, with no interface in it, still has no address to probe. Needs an endpoints file.

Recorded against: YouTube, Verizon, Facebook

Imported from docs/replays/youtube-pakistan-2008/FACTS.md and the replay manifests in the Hammerhead repository at commit 0ee9732.