HAProxy DDoS Hardening and Tuning

HAProxy is frequently the first thing an attack reaches that can make a decision, which makes it the natural place to apply connection-level controls. It is also, for the same reason, the component whose defaults most often decide whether a moderate flood is an incident.

Everything below is about the proxy itself. The circuit ceiling is unchanged by any of it.

Establish the baseline first

# Live counters, per frontend, backend and server
echo "show stat" | socat stdio /var/run/haproxy.sock | cut -d, -f1,2,5,8,34,35

# Current session rate and totals
echo "show info" | socat stdio /var/run/haproxy.sock | grep -E 'CurrConns|SessRate|MaxSessRate|CumConns'

# What the stick tables currently hold
echo "show table" | socat stdio /var/run/haproxy.sock

Record MaxSessRate and CurrConns across a period that includes a genuine business peak. Every limit below is set against those numbers rather than against a default.

Connection ceilings

global
    # Process-wide ceiling. Memory is roughly this × per-connection cost;
    # size it against measured RAM rather than optimistically.
    maxconn 60000

defaults
    mode http
    option httplog

frontend web
    bind :443 ssl crt /etc/haproxy/certs/
    # Frontend ceiling: what HAProxy itself will accept.
    maxconn 50000
    default_backend origin

backend origin
    # Per-server ceiling: what actually reaches the origin. Requests beyond
    # this queue in HAProxy rather than overwhelming the backend.
    server app1 10.0.0.11:8080 check maxconn 400
    server app2 10.0.0.12:8080 check maxconn 400
    # Bound the queue too — an unbounded queue converts an overload into
    # a latency collapse that looks like an outage anyway.
    timeout queue 5s

The pattern that matters: a high frontend ceiling with a much lower per-server ceiling, and a bounded queue between them. That converts a surge into added latency and shed load rather than into a backend failure.

Timeouts — the slow-HTTP answer

defaults
    # The decisive one: how long a client may take to send complete headers.
    # Slowloris-class attacks exist entirely in the space this leaves open.
    timeout http-request    5s

    # Idle client connection.
    timeout client          30s
    # Half-closed client connection still consuming a slot.
    timeout client-fin      10s

    # Connecting to the backend, and backend idle.
    timeout connect         5s
    timeout server          30s
    timeout server-fin      10s

    # Keep-alive reuse window; shorter under pressure.
    timeout http-keep-alive 10s
    # Time a request may sit in the queue before being rejected.
    timeout queue           5s
    # Tarpit duration for requests you want to slow rather than reject.
    timeout tarpit          30s

Verify the effect rather than assuming it:

# Open a connection and send headers one byte at a time; it should be
# closed after timeout http-request rather than held open.
printf 'GET / HTTP/1.1\r\nHost: example\r\n' | timeout 20 nc -q 20 127.0.0.1 443

Stick tables: per-source rate control

frontend web
    bind :443 ssl crt /etc/haproxy/certs/

    # One table, several counters, tracked per source address.
    # 1m expiry keeps memory bounded; size against your real client count.
    stick-table type ip size 1m expire 1m store \
        conn_cur,conn_rate(10s),http_req_rate(10s),http_err_rate(10s)

    # Track every incoming connection against it.
    http-request track-sc0 src

    # Concurrent connections from one source.
    http-request deny deny_status 429 if { sc0_conn_cur gt 60 }

    # New connections per 10s from one source.
    http-request deny deny_status 429 if { sc0_conn_rate gt 200 }

    # Requests per 10s from one source.
    http-request deny deny_status 429 if { sc0_http_req_rate gt 400 }

    # Error rate — a source generating mostly 4xx is probing, not browsing.
    http-request deny deny_status 429 if { sc0_http_err_rate gt 100 }

Inspect what the table actually holds during an event:

# Sources currently over a threshold
echo "show table web data.http_req_rate gt 400" \
  | socat stdio /var/run/haproxy.sock

# Clear one entry after a false positive
echo "clear table web key 203.0.113.9" | socat stdio /var/run/haproxy.sock

The per-source caveat applies here as everywhere. Carrier-grade NAT means one address can legitimately represent thousands of users, so these thresholds have to be set well above what a shared egress produces, or they will block a mobile network before they inconvenience an attacker. The reasoning is worked through in rate limiting.

Protecting the expensive endpoints specifically

frontend web
    # A second table keyed on the path group rather than the client,
    # so the limit protects the operation rather than punishing a user.
    stick-table type string len 64 size 100k expire 10s store http_req_rate(10s)

    acl expensive path_beg /search /report /export
    http-request track-sc1 base32 if expensive
    http-request deny deny_status 429 if expensive { sc1_http_req_rate gt 200 }

Slowing rather than rejecting

    # Tarpit holds the connection instead of returning immediately, which
    # costs the attacker a slot. Use sparingly — it costs you one too.
    http-request tarpit if { sc0_http_req_rate gt 800 }

    # Silently drop, no response at all.
    http-request silent-drop if { sc0_conn_rate gt 500 }

silent-drop is worth understanding before enabling: it closes without a TCP reset, which costs the attacker a timeout rather than an immediate signal, and equally leaves a legitimate client hanging if the rule was wrong.

TLS-specific pressure

frontend web
    bind :443 ssl crt /etc/haproxy/certs/ \
        ciphers ECDHE+AESGCM:ECDHE+CHACHA20 \
        no-sslv3 no-tlsv10 no-tlsv11

    # Handshake rate per source: TLS renegotiation and handshake floods
    # consume CPU rather than bandwidth.
    stick-table type ip size 1m expire 1m store gpc0,conn_rate(10s)

# Watch handshake cost during a test
openssl s_time -connect 127.0.0.1:443 -new -time 10

Observability

frontend stats
    bind 127.0.0.1:8404
    stats enable
    stats uri /stats
    stats refresh 5s

# Rejections by reason, from the log
awk '{print $11}' /var/log/haproxy.log | sort | uniq -c | sort -rn | head

# Queue depth and denied counters
echo "show stat" | socat stdio /var/run/haproxy.sock \
  | awk -F, 'NR==1 || $1=="origin" {print $1,$2,$3,$4,$5,$6}'

Track dreq and dresp — denied requests and responses — as first-class metrics. A rise in them during normal traffic is a false-positive incident, not a success.

Safe rollout order

1. Set timeouts first. They carry the lowest false-positive risk and answer the slow-HTTP class immediately.
2. Add maxconn at server level, then backend, then frontend, measuring queue depth between each.
3. Add stick tables in observe mode — track without denying — and watch the distribution for at least one business peak.
4. Only then convert the observed thresholds into denials, set well above the peak you measured.
5. Keep clear table in the runbook, because the first false positive will need it.

Rollback

# Validate before applying — a bad config that reloads is worse than one that fails
haproxy -c -f /etc/haproxy/haproxy.cfg

# Seamless reload, existing connections preserved
systemctl reload haproxy

# Revert
cp /etc/haproxy/haproxy.cfg.bak /etc/haproxy/haproxy.cfg && \
  haproxy -c -f /etc/haproxy/haproxy.cfg && systemctl reload haproxy

The honest limit

Every control on this page acts after the packet has arrived. HAProxy decides what happens to a connection, and it decides well — but a flood that fills the circuit feeding it never reaches a decision point. Host and proxy tuning raise the floor; the ceiling is upstream, which is the argument the pillar makes at length and the firewall boundary page makes for each control in turn.