BGP Leak Cascades: How a Single Routing Mistake Takes Down Perfectly Healthy Cloud Clusters

A few years back I remember sitting there refreshing Instagram, convinced my phone was broken, while half the internet quietly lost its mind. WhatsApp wouldn’t send. Facebook wouldn’t load. And the strange part, once the postmortems came out, was that none of it was caused by a fire, a hack, or a hardware failure. It came down to a routing mistake — the internet’s equivalent of a single wrong turn that somehow closed every road into a city at once.

That’s the part people find hardest to believe about the cloud. We’re told it’s basically indestructible — if a data centre in Virginia goes dark, your app should quietly reroute to Frankfurt without you ever noticing. Redundant servers, mirrored databases, engineers sleeping soundly. Losing a server, or even a whole building, isn’t supposed to take anything down.

And yet every so often, a big chunk of the internet just goes dark. Streaming stutters, banking apps throw errors, smart home gadgets stop responding. Engineers dig in and find something odd: every server is perfectly healthy, sitting at 100%.

So what actually broke? The digital roads leading to those servers simply vanished.

This is the strange, slightly chaotic world of BGP route leaks — and once you understand how they work, a lot of “mystery outages” stop being mysterious.

What Is BGP, in Plain English?

How an unauthorised BGP announcement propagates across Autonomous Systems (AS), causing unintended traffic redirection.

To understand how the internet breaks, it helps to understand how it finds its way around in the first place.

Picture driving across a country with no GPS. Every time you cross a border, you have to stop and ask a guard, “what’s the way to the capital?” The guard hands you a map and says, “take this highway north.”

That guard is basically doing the job of BGP — the Border Gateway Protocol.

The internet isn’t one giant machine — it’s thousands of independently run networks stitched together: ISPs, cloud companies, universities, mobile carriers. Each one is called an Autonomous System, or AS for short. Since there’s no single master map of the internet, these networks are constantly telling each other where things are, using three basic ideas:

  • IP prefixes — the block of addresses a network claims as “mine,” a bit like a postcode.
  • BGP announcements — messages that say, “send me anything addressed to this block, and I’ll get it there.”
  • Routing tables — the address book each router builds from those messages, so it knows the fastest way to send your data.

So when people talk about a “routing table failure,” nothing physically breaks. No router explodes. It’s simply that the address book got updated with bad directions — and every router that trusted it now believes something false.

What Is a BGP Route Leak, Really?

Network engineers have a running joke that the whole internet is held together by duct tape and trust. That trust is exactly what BGP relies on. When a network announces a route, its neighbours generally take it at face value and pass it along.

It’s worth separating two things people often lump together:

  • A BGP hijack is deliberate — someone putting up a fake road sign to steal your traffic.
  • A BGP route leak is (usually) an honest mistake — correct directions, given to the wrong audience.

The airport analogy I keep coming back to: imagine a gate agent in Chicago who knows that passengers flying to Tokyo can connect through Dubai. Correct information. Now imagine that same agent grabs the wrong microphone and announces to the entire terminal that every single flight, to every city on earth, should now connect through Dubai.

Dubai’s runway is real. The route isn’t fake. But funnelling the whole planet’s air traffic through one hub causes instant, total gridlock — and that’s essentially what a route leak does to internet traffic.

How One Bad Route Turns Into a Global Cascade

Here’s the part that still gets me: it doesn’t take a big mistake to cause a big outage. It takes a small one, at the wrong layer, at the wrong time.

Picture an engineer tweaking a router filter late at night, right at the tail end of a long shift. One line in the config gets deleted — maybe it was meant to retire an old route — and suddenly the filter is far more permissive than intended.

  • Network A now advertises an internal-only route out to Network B.
  • Network B sees what looks like a perfectly good shortcut and passes it on to everyone else.
  • Within minutes, routers on five continents have updated their maps with this “shortcut.”
  • Traffic that used to spread across the globe suddenly funnels down a single pipe that was only ever built for local, low-volume traffic.
  • That pipe floods instantly. Packets queue up, get dropped, or get quietly rejected by firewalls that notice the traffic is arriving from a direction it was never supposed to come from.

This isn’t a hypothetical. On 22 January 2026, Cloudflare had exactly this happen. A routine change meant to stop their Miami site from advertising a set of IPv6 addresses ended up making the export policy too loose — instead of blocking those routes, it let them all through. For 25 minutes, Miami was broadcasting internal routing information to the rest of the internet, other networks trusted it, and roughly 12 gigabits per second of traffic got dropped worldwide before Cloudflare’s own engineers spotted it and rolled the change back by hand. No hack, no outage at the server level — just one over-permissive line in a routing policy.

Why “Spreading Your Servers Around” Doesn’t Fully Protect You

The standard advice in cloud architecture is geographic redundancy: spread your systems across regions, so that if one goes down, the others pick up the slack.

The catch that a lot of people miss: spreading your servers out geographically doesn’t mean your routing is spread out too. Plenty of multi-region setups still lean on shared infrastructure underneath — the same public IP ranges, the same transit providers, sometimes even the same routing policy applied everywhere.

Anycast is a good example of how this bites people. With Anycast, servers on three continents can all sit behind the exact same public IP address, and BGP is supposed to send each user to whichever copy is physically nearest to them. It’s a neat trick — until the routing information lies. If a route leak convinces the internet that the European node is suddenly the “closest” option for everyone, users in Tokyo, Sydney, and Toronto all get shipped across the globe to Europe at once.

This actually happened, and it’s a good illustration of the scale involved. Back in 2014, a Latin American ISP called Internexa accidentally announced that it — rather than Cloudflare — was the right path to Cloudflare’s global network. For 49 minutes, huge volumes of traffic meant for data centres all over the world got funnelled into a single facility in Medellín, Colombia, which was never built to handle that kind of load. Traffic to Cloudflare-protected sites dropped by around half in North America, even though every actual Cloudflare data centre elsewhere was working perfectly.

How a Perfectly Healthy Server Can Look Completely Offline

If you’ve ever worked anything resembling an ops or support role, you’ll know this feeling: the dashboard is all green, every check is passing, and yet the complaints are pouring in.

A perfectly working server is useless if nothing can find its way there and back. Two things tend to happen:

  • Black-holing — a router receives a bad route, has nowhere sensible to send the traffic, and just drops it. No error, no bounce-back — the data simply disappears.
  • Patchy, regional outages — because different providers pick up and process the bad route at different speeds, you get weird pockets of failure. Someone in one city sees a broken page while someone twenty miles away loads the site fine.

In warehouse operations — which is closer to my own day-to-day than networking — we’d call this a “phantom stockout.” The item is physically sitting on the shelf, the system says it’s in stock, but somewhere between the shelf and the customer, the path is broken. The stock is fine. The route to it isn’t. It’s oddly the same failure, just made of cardboard instead of packets.

Why Automatic Failover Can Make Things Worse, Not Better

Modern cloud systems are built to self-heal: if the primary region stops responding, traffic automatically shifts to a backup. Good instinct — except during a route leak, this can actively backfire.

Here’s the chain reaction:

  1. Health checks notice Region A has “stopped responding” — really, its traffic just can’t get there anymore.
  2. The failover system assumes Region A is dead and shifts all traffic to Region B.
  3. Region B, which may already be straining under the same route leak, gets hit with a sudden flood it was never sized for — and buckles.
  4. Someone spins up more servers to cope. It doesn’t help, because the problem was never the servers.

One routing typo, and you get a full chain: leak → dropped packets → automatic failover → overloaded backup.

How Networks Actually Guard Against This

None of this means engineers just shrug and hope for the best. There are real, layered defences:

  • Route filtering — explicitly defining which addresses a neighbouring network is even allowed to announce.
  • RPKI (Resource Public Key Infrastructure) — a cryptographic way to check whether a network is actually authorised to claim a given block of addresses.
  • Maximum-prefix limits — an automatic circuit breaker that disconnects a neighbour if it suddenly starts announcing an implausible number of routes.
  • Staged rollouts — testing a routing change on a small slice of the network before pushing it everywhere at once.

RPKI is great at stopping someone from impersonating your addresses outright. But it doesn’t stop a legitimate network from accidentally sharing a real route with the wrong people — that still comes down to careful configuration and someone double-checking the change before it goes live everywhere at once.

The Bigger Lesson: Redundancy Has to Include the Network Itself

Real cloud resilience is more than duplicated databases and extra server instances scattered across availability zones. If all of that redundancy still depends on one shared routing policy, or one BGP link somewhere upstream, you don’t actually have a backup — you have several copies of the same single point of failure.

The question worth asking isn’t just “do we have a backup server running?” It’s “if the main path goes down, can a user’s request actually get to that backup through a genuinely separate route?”

Funnily enough, that’s the same question I ask on the warehouse floor when a conveyor jams: it’s not enough to have a second line running somewhere — it only helps if there’s an actual, working path to get things onto it. The internet, it turns out, has the exact same problem, just moving at the speed of light instead of the speed of a forklift.

Scroll to Top