Linux Network Stack Tuning for DDoS: NIC Queues, RSS/RPS and XDP

This guide is the packet-rate companion to the Linux server hardening guide. That guide defends the connection state — SYN queues, conntrack, descriptors — with sysctl. This one defends the packet-processing path — the NIC, its queues, the interrupts and the receive path — when the attack is measured in packets per second rather than in state.

The symptom that sends you here is specific: one CPU core pinned at or near 100% in softirq while the others idle, with bandwidth nowhere near the circuit limit. A flood of small packets can saturate a single core’s packet processing long before it fills the pipe, and no sysctl in the other guide touches it. Every setting below comes with its ethtool or sysfs command and the counter that shows it working.

Appliance	Symptom	Layer	Command
One core at 100% softirq, others idle	RSS / RPS steering, IRQ affinity	ethtool -L; /sys .../rps_cpus; /proc/irq
Rising softnet 'dropped' column	NIC ring buffer; netdev backlog/budget	ethtool -G; sysctl net.core.netdev_*
Rising softnet 'squeezed' column	softirq budget; interrupt coalescing	netdev_budget; ethtool -C
High pps of junk reaching the stack	XDP drop at the driver, before conntrack	ip link set ... xdp; a small XDP program
Flow locality lost across cores	RFS (receive flow steering)	rps_flow_cnt; rps_sock_flow_entries

Every row is the driver/interrupt layer, below the sysctl hardening in the Linux server guide. The symptom that sends you here is CPU in softirq with bandwidth nowhere near the limit. None of it helps once the circuit itself is full.

0. Baseline first: read where the packets and the CPU are

The single most important reading is per-core softirq time and the softnet statistics:

# Per-core softirq — is one core hot while others idle?
mpstat -P ALL 1 3        # watch the %soft column per CPU
# or
watch -n1 "grep . /proc/softirqs | head"

# softnet_stat: col1 processed, col2 DROPPED, col3 time/budget SQUEEZED (per CPU, hex)
cat /proc/net/softnet_stat

# Which IRQs are firing, and on which CPU
cat /proc/interrupts | grep -Ei 'eth|ens|enp|mlx|i40e|ixgbe'

# NIC-level drops and errors
ethtool -S eth0 | grep -Ei 'drop|miss|error|fifo|rx_no_buffer'

A single hot %soft core with the others idle is the packet-rate signature. The softnet_stat second column (drops) and third column (squeezed) tell you whether the backlog or the budget is the constraint. These are the baselines every change below is judged against.

1. NIC ring buffers: the first place packets are lost

The receive ring buffer is where the NIC deposits packets for the kernel to pick up. Under a packet flood, an undersized ring drops packets before any steering can help:

# Current and maximum ring sizes
ethtool -g eth0
# Raise RX (and TX) toward the maximum the NIC supports
ethtool -G eth0 rx 4096 tx 4096

# Confirm the drop counter stops climbing after the change
ethtool -S eth0 | grep -Ei 'rx_no_buffer|rx_missed|fifo'

A larger ring costs a little memory and latency; on a server taking a packet flood, that trade is almost always worth it. If rx_no_buffer_count keeps climbing at the maximum ring size, the constraint has moved to the CPU, which is the next sections.

2. RSS: spread receive processing across cores in hardware

Receive Side Scaling hashes incoming flows across multiple NIC receive queues, each with its own interrupt on its own core. It is the most efficient way to stop a single core being the bottleneck:

# How many combined/receive channels does the NIC have, and how many are enabled?
ethtool -l eth0
# Enable as many combined queues as you have cores to service them
ethtool -L eth0 combined 8

# Show the RSS indirection table (which queue each hash bucket maps to)
ethtool -x eth0

# After enabling, /proc/interrupts should show the NIC IRQs spread across cores,
# and mpstat should show %soft spread rather than one hot core
mpstat -P ALL 1 3

RSS is the first choice because the spreading happens in hardware at no CPU cost. Its limit is the number of queues the NIC provides; where that is too few — common on virtualised NICs — RPS fills the gap in software.

3. RPS and RFS: software steering where hardware falls short

Receive Packet Steering spreads packets across cores in the kernel; Receive Flow Steering adds flow locality so a flow lands on the core running its application:

# RPS: set the CPU mask (hex bitmask of cores) for a receive queue
echo ff > /sys/class/net/eth0/queues/rx-0/rps_cpus

# RFS: global flow table size, then per-queue
echo 32768 > /proc/sys/net/core/rps_sock_flow_entries
echo 32768 > /sys/class/net/eth0/queues/rx-0/rps_flow_cnt

# Verify the spread moved: no single core should carry all the softirq load now
grep NET_RX /proc/softirqs

Enable RPS per receive queue with a mask that excludes the cores you reserve for the application. RFS on top keeps a flow’s packets going to one core, which helps cache locality — set rps_sock_flow_entries and the per-queue rps_flow_cnt together.

4. IRQ affinity: pin the queues, consider disabling irqbalance

With RSS queues in place, pinning each queue’s interrupt to a dedicated core makes the spread predictable instead of letting irqbalance move it under load:

# Find the NIC's per-queue IRQ numbers
grep -Ei 'eth0|ens|enp' /proc/interrupts | awk '{print $1}' | tr -d ':'

# Pin an IRQ to a specific core (echo the core's hex mask to smp_affinity)
echo 2 > /proc/irq/145/smp_affinity     # core 1
echo 4 > /proc/irq/146/smp_affinity     # core 2

# On a dedicated high-rate host, stop irqbalance from re-shuffling them
systemctl stop irqbalance
systemctl disable irqbalance

Whether to disable irqbalance is a measurement decision, not a universal one: on a dedicated host under packet-rate attack, manual pinning is more predictable; on a general-purpose server, irqbalance is usually fine. Reserve the cores carrying NIC interrupts, and keep the application off them.

5. softirq budget and interrupt coalescing

If the third softnet_stat column (squeezed) climbs, the softirq handler is hitting its budget and yielding before draining the queue. Raise the budget, and use coalescing to cut per-packet interrupt overhead:

# Raise how many packets and how long a softirq pass may process
sysctl -w net.core.netdev_budget=60000
sysctl -w net.core.netdev_budget_usecs=8000
sysctl -w net.core.netdev_max_backlog=250000

# Interrupt coalescing: batch interrupts; adaptive raises it as rate climbs
ethtool -C eth0 adaptive-rx on rx-usecs 64

# The squeezed column (3rd per row) should stop climbing
awk '{print strtonum("0x"$3)}' /proc/net/softnet_stat

Coalescing trades a little latency for a lot of efficiency under a flood; measure the latency- sensitive path in normal operation before committing to an aggressive rx-usecs.

6. Offloads: what to keep and what to drop under attack

Stateless offloads (checksum, GRO/GSO/TSO) generally help and should stay on. The one to scrutinise is LRO, which merges packets in a way that can interfere with forwarding and with some inspection:

# Show offload state
ethtool -k eth0 | grep -Ei 'gro|gso|tso|lro|rx-checksum'
# Generic Receive Offload usually helps; keep it on
ethtool -K eth0 gro on
# LRO off if the host forwards/bridges or you need per-packet visibility
ethtool -K eth0 lro off

7. XDP: drop at the driver, before the stack

For a packet-rate flood that the steering layer cannot spread away, XDP is the one host tool that changes the order of magnitude. It runs a small program in the driver, before the packet enters the network stack — before conntrack, before nftables, before allocation — and can drop millions of packets per second at a fraction of the CPU the same drop costs later:

# Attach a compiled XDP program (drops by whatever criterion it implements)
ip link set dev eth0 xdp obj xdp_drop.o sec xdp

# Confirm it is attached and in which mode (native is best; generic is the fallback)
ip link show eth0 | grep -o 'xdp[a-z]*'

# Detach
ip link set dev eth0 xdp off

Native XDP needs driver support; without it the program runs in generic mode with less benefit. XDP is more work than the other sections and is not the first reach — but when packets per second is the attack and steering has run out, dropping in the driver is what a dedicated appliance does in hardware, done in software at the host edge.

The signals to wire into monitoring

# 1. Per-core softirq — the hot-core signature
mpstat -P ALL 1 1 | awk '/%soft|Average/{print}'
# 2. softnet drops and squeezes
awk '{d+=strtonum("0x"$2); s+=strtonum("0x"$3)} END{print "dropped",d,"squeezed",s}' /proc/net/softnet_stat
# 3. NIC-level drops
ethtool -S eth0 | grep -Ei 'rx_no_buffer|rx_missed|drop'
# 4. Total interrupt rate
grep NET_RX /proc/softirqs

A hot single core with a climbing squeezed count is the packet-rate signature; NIC drops climbing at a maximised ring buffer means the CPU, not the ring, is the bottleneck and steering or XDP is the next move.

The honest limit of this layer

This layer defends against packet rate — packets per second saturating CPU — which can strike well below the bandwidth limit. Two cases sit outside it.

The first is bandwidth: if the attack fills the access circuit, packets are dropped upstream before the NIC sees them, and no queue, steering or XDP setting applies.

The second is state exhaustion, which is the other guide’s subject — connection tables and queues, not packet processing. The two guides together cover the host: this one keeps the CPU able to process packets, the server hardening guide keeps the connection state from filling. Above the circuit, the answer is the network tier, where the same drop this guide does in XDP is done in hardware at scale.

Order of application

1. Confirm the symptom (Section 0): one hot softirq core, bandwidth well under the limit.
2. Raise the NIC ring buffers; confirm NIC drops stop.
3. Enable RSS with as many queues as cores can service; verify the spread.
4. Add RPS/RFS where hardware queues are too few.
5. Pin IRQ affinity; decide on irqbalance from measurement.
6. Raise the softirq budget and enable adaptive coalescing if squeezes persist.
7. For a flood the steering cannot spread, deploy an XDP drop program.
8. Wire the four signals into monitoring.

Steps 3 and 4 solve most packet-rate problems by spreading the load; XDP in step 7 is the escalation for what spreading alone cannot absorb.