Stateful and Stateless DDoS Defence: What Runs Out First

Every security device in a traffic path has to decide how much it remembers. That decision looks like an implementation detail and is in fact the property that determines how the device behaves on its worst day.

The reason is an asymmetry. To track a conversation, a device must allocate memory the moment the conversation appears to start — before it knows whether the conversation is real.

At a glance

Appliance	Property	Stateless filtering	Stateful inspection
Cost per packet	Fixed, and small	Variable, and includes memory that persists
What an attacker must spend	As much as you spend	Far less than you spend
What it can decide	Header properties, rates, source and protocol	Whether a packet belongs to a legitimate conversation
Failure mode under load	Drops at line rate; behaviour stays predictable	Table fills, then admission stops or degrades
Where it belongs	In front, facing the internet	Behind, seeing traffic that has already been reduced

Neither column is the better technology. The right reading is that they fail differently, and that the order in which they are placed decides which failure you get.

The asymmetry, in one exchange

An attacker sends a TCP SYN with a forged source address. It costs them one small packet and no memory, because they will never track the result.

The firewall receiving it must create a session-table entry: source, destination, ports, sequence numbers, a timer. That entry occupies memory until it expires, typically after seconds.

One cheap action on their side, one expensive and persistent allocation on yours. Multiply by the packet rate a modest circuit can deliver and the table fills long before the bandwidth graph looks unusual. This is the whole mechanism, and RFC 4987 documented it decades ago.

What happens when the table is full is a design decision that varies by product: some refuse new sessions, which denies service to legitimate users; some overwrite the oldest entries, which breaks live conversations; some pass traffic uninspected to stay available. All three are defensible engineering choices and all three mean the control is no longer doing its job.

Why this makes the firewall a poor first line

The firewall is not badly built. It is being asked to do something its architecture forbids.

A firewall’s value comes from understanding conversations — that is what allows it to enforce policy rather than blindly filter. Understanding conversations requires state. Requiring state creates the asymmetry above. The capability and the vulnerability are the same property.

There is a second effect that shows up on the invoice rather than the dashboard. Because the firewall must be sized for the worst packet rate that can reach it rather than for the policy work it actually does, organisations buy far larger firewalls than their rulebase needs. That economics is worked through in why an on-premises tier lowers TCO, and the control-ownership question in firewall, IPS, WAF or DDoS appliance.

What stateless filtering can and cannot do

Stateless filtering decides on what is visible in the packet: source address, protocol, ports, flags, length, and rate. It holds nothing between packets, so its cost per packet is fixed and its behaviour under load is predictable. It drops at line rate and keeps dropping.

That predictability is the point. A stateless stage cannot be exhausted the way a table can, because there is nothing to exhaust.

What it cannot do is know whether a well-formed packet belongs to a legitimate conversation. It cannot tell a genuine user’s HTTP request from an identical-looking hostile one. Anything requiring that judgement requires state, or requires the application layer, and neither is free.

Several established mechanisms live in this stateless category: ingress filtering removes spoofed sources at the edge, rate limiting bounds what any source may consume, and FlowSpec pushes packet-property filters upstream to where there is more room.

Ordering, which is the actual design decision

The useful principle is not stateless is better. It is:

Do the cheap work first, and only allocate state for what survives it.

A path ordered that way makes the attacker pay before your expensive resource is touched. Malformed packets, spoofed sources, and traffic from protocols you do not serve are removed at fixed cost. Rate limits bound what any single source can present. Whatever is left — a much smaller volume, consisting of traffic that at least looks legitimate — reaches the stateful stage, which now has room to do the job it is good at.

Reverse the order and the stateful stage meets the full attack, which is the arrangement most estates have by default because the firewall was installed first and the mitigation layer was added later, behind it.

The one place state is unavoidable

Application-layer attacks defeat stateless defence by design. A flood of well-formed HTTP requests from real hosts is indistinguishable from popularity on a per-packet basis. Separating them requires knowing something about the session, the client’s behaviour over time, or the application’s own semantics.

So the layered answer is not stateless everywhere. It is stateless at volume, stateful where judgement is required, and — critically — the second stage protected from having to see the first stage’s traffic. SYN proxy and SYN cookies is the classic hybrid: a front stage completes the handshake without allocating per-connection memory, and only hands the connection back once the client has proved it exists.

Establishing your own numbers

Three measurements, none of which requires a vendor:

1. Session table utilisation at genuine peak, on every stateful device in the path. The gap between that and the ceiling is the room an attacker has.
2. Session establishment rate at peak. Compare against the manufacturer’s stated maximum, which is often much closer to your normal figure than the concurrent-session number suggests.
3. Behaviour at the ceiling, from documentation or from a test: refuse, overwrite, or pass.

The third is worth insisting on in writing. It is the difference between a degraded service and an open one, and it is rarely on a datasheet — one of the questions the tender list puts in Section 2 for exactly this reason.