When "idle" isn't idle: how a Linux kernel optimization became a QUIC bug

CUBIC, the default congestion controller in Linux and quiche, can become permanently stuck at its minimum congestion window after an early congestion collapse. Cloudflare found the bug in a QUIC test where packet loss stopped completely, yet CUBIC continued oscillating between recovery and congestion avoidance instead of increasing its sending rate. The problem was traced to a Linux TCP optimization for idle or app-limited connections, and ultimately fixed with an elegant near-one-line change.

How CUBIC manages traffic

• CUBIC controls the sender’s congestion window (cwnd), limiting how many bytes can be in flight.
• It increases cwnd when acknowledgments arrive without loss and reduces it when loss suggests the network is overloaded.
• As quiche’s default congestion controller, CUBIC affects a substantial amount of QUIC traffic.
• Recovery from the minimum congestion window is an important but relatively under-tested part of congestion control.

The failing test

• The test downloaded a 10 MB file over HTTP/3 between local quiche client and server.
• Network conditions included:
  • 10 ms RTT
  • 30% random packet loss during the first two seconds
  • No packet loss after two seconds
  • A 10-second timeout
• The expected result was for CUBIC to reduce its window during loss, then steadily recover once the network became reliable.
• Instead, approximately 60% of repeated 100-run test batches failed to finish in time.

CUBIC becomes stuck at its minimum

• After packet loss stopped at two seconds, bytes in flight remained flat rather than increasing.
• CUBIC’s congestion window stayed at its minimum of 2,700 bytes—roughly two full-sized packets.
• The controller repeatedly switched between recovery and congestion avoidance:
  • 999 transitions over about 6.7 seconds
  • One transition approximately every 14 ms
• The oscillation closely matched the connection’s RTT, indicating that each ACK round was triggering the behavior.
• Because the test was a download, client ACKs caused the server’s bytes in flight to fall to zero; the server then sent another two-packet burst, repeatedly provoking the faulty state transition.
• Reno passed the same test 100% of the time, confirming that the issue was specific to CUBIC rather than the test setup.

The connection to Linux TCP

• The investigation focused on behavior when bytes_in_flight == 0, effectively an idle or app-limited condition.
• A 2017 Linux kernel change addressed a TCP CUBIC issue after application idle periods.
• Before the change, CUBIC’s epoch could remain unchanged for a long time while the application was idle.
• When sending resumed, the elapsed time used by CUBIC could be extremely large, producing an excessively aggressive growth slope and dangerous congestion-window inflation.
• The kernel optimization was intended to align CUBIC with the app-limited exclusion described in RFC 9438 §4.2-12.
• Porting this logic to QUIC exposed an unintended interaction: repeated short periods with no bytes in flight could be interpreted incorrectly, causing CUBIC to cycle between states and remain at its minimum window.

The practical lesson is that congestion-control implementations must test not only steady-state throughput and ordinary loss recovery, but also recovery from the minimum window and repeated app-limited or idle periods. In this case, a small adjustment to the idle-state handling broke the cycle and allowed CUBIC to recover normally.