Under a sustained flood of malicious requests, blocking the source IP felt like the obvious first move, and it was the right first move. It also wasn’t the whole fix, and the gap took a little longer to notice than the attack itself did.
Why the block didn’t actually stop anything
Adding an offending IP to a deny rule at the edge is normally enough, because a normal client’s request really does originate from the IP you see. That assumption breaks the moment traffic passes through any kind of intermediary, proxy, CDN, or forwarding layer before it reaches your edge, because the visible source IP at that point isn’t necessarily the attacker’s IP anymore. It’s whatever hop last touched the connection. An attacker routing through a proxy shows up, from the perspective of a plain source-IP block, as an entirely different, unblocked address every time they rotate proxies. The block was correctly enforcing exactly what it said it would enforce. It just wasn’t enforcing the thing anyone actually needed blocked.
The traffic itself gave this away eventually: the pattern, payload, and timing of new requests kept matching the same attack, from IPs that had never appeared before, immediately after the original address got blocked. That’s not a new attacker discovering the same vulnerability independently. That’s the same attacker, one hop further back than the block was looking.
There’s a header most proxies and load balancers use specifically to preserve the original client address as traffic passes through: it gets prepended with each hop, so in principle the real source is still recoverable. But it’s just a header, set by whatever sent the request, and nothing forces it to be honest. An attacker in direct control of the request can put anything they want in it, including a value chosen specifically to look legitimate or to blend in with normal traffic.
It’s worth sitting with why the first block felt sufficient at the time. Source-IP blocking is the default reflex because it’s usually correct, and correcting for every possible network path in front of every rule would be exhausting overkill for the vast majority of traffic that never needs it. The mistake wasn’t choosing source-IP blocking first. It was treating the first block as the end of the response instead of a first pass, and not immediately checking whether the exact same attack pattern kept showing up under a new address right after.
Blocking on two signals instead of one
The fix layered a second rule on top of the first: also inspect that forwarded-for header for the known-bad address, positioned to run before the plain source-IP check, and block on either matching. That covers both cases, the client hitting the edge directly and the client routing through something that forwards the header, without depending on either signal alone being trustworthy or complete.
It’s worth being deliberate about which position in that header actually matters, since a chain of proxies can append more than one address to it, and picking the wrong position (or trusting the whole thing blindly) reopens the exact problem you’re trying to close: treating attacker-controlled data as ground truth. And it’s worth deciding explicitly what happens to traffic that has no such header at all, since blocking on its absence would just take down every legitimate direct connection instead of the traffic you’re actually after.
What this generalizes to
A block rule is only as strong as the assumption it’s built on about where the identifying signal comes from, and “the client’s real address is whatever field says so” is an assumption, not a fact, the instant anything sits between the client and your edge. Any access-control decision keyed on a single piece of request data is a decision the other side can route around, cheaply, the moment they notice it exists. The useful question isn’t “did we block the IP.” It’s “what has to be true about the network path for this specific block to actually mean what we think it means,” and that question is worth asking before an incident forces it, not during one.
The same shape of problem shows up anywhere a system trusts a piece of data about the request’s origin without asking who actually controls that data. A geolocation check based on an IP, a device fingerprint based on a client-supplied header, an authentication decision based on anything the caller gets to set: each of those is only as trustworthy as the layer that produced it, and each one is worth a quick mental test before you rely on it for a security decision. If the party you’re trying to defend against is also the party who gets to write the value you’re checking, that value was never really evidence in the first place. It was just a convenient assumption that happened to hold, right up until someone specifically decided to test it.