What is a TLS handshake flood attack?

Cryptography is the only part of HTTPS that is expensive

Serving a cached page over an established connection is close to free: a symmetric cipher runs at gigabytes per second per core and the content comes out of memory. Starting a new HTTPS connection is a different kind of work. Before any request exists, your server has to:

1. Complete the TCP handshake (cheap, but see SYN floods for what happens when even this is attacked).
2. Generate an ephemeral key share and compute a shared secret.
3. Sign something with your certificate's private key, to prove the certificate is yours.
4. Send the certificate chain — typically three to six kilobytes of it.

Step 3 is the expensive one, and the order matters more than the cost: in both TLS 1.2 with ECDHE and TLS 1.3, the server performs that signature in the flight it sends immediately after the ClientHello — before it has any evidence that the client on the other end intends to finish, or exists at all.

Do not take those numbers from a web page: run openssl speed rsa2048 ecdsap256 ecdhx25519 on the machine that actually terminates your TLS. The result, divided by your core count and halved for everything else the box is doing, is your real ceiling in new HTTPS connections per second. For a great many origins that number lands somewhere between one and ten thousand — which is well within the reach of a rented botnet, and nowhere near the gigabits those same providers advertise protection against.

The asymmetry, written out

Read the first row twice. Even ignoring CPU entirely, an abandoned handshake makes you transmit roughly eight times what the attacker transmitted, which quietly turns a modest inbound flood into a meaningful outbound one. This is why egress often rises faster than ingress during a handshake flood — a detail that misleads people into looking for a scraper.

Six shapes of the same attack

The last row is the one defenders usually miss. Your TLS configuration is a menu, and the attacker orders from it. If a large finite-field Diffie-Hellman group or an RSA key-exchange suite is still on that menu for compatibility, a ClientHello advertising nothing else will be served exactly that.

The first three rows are also where a handshake flood blurs into neighbouring attack families. One request per connection at scale is simultaneously a Layer 7 attack and a crypto-exhaustion attack; an abandoned handshake that is deliberately kept half-finished for as long as your timeout allows is a low-and-slow attack that happens to be aimed at the TLS layer. Real incidents rotate between them, which is the subject of multi-vector attacks.

Your bandwidth graph will mislead you again

The arithmetic is the same lesson as a SYN flood, one layer up:

Forty megabits per second will not register on a dashboard calibrated for gigabits. If your monitoring plots bandwidth and requests per second but not new connections per second, a handshake flood is close to invisible to you until the site is already down, and the symptom — high CPU, HTTPS timing out, access log quieter than usual — reads exactly like a server fault.

Confirming that this is what is happening

Five checks, in the order that settles it fastest:

• Profile the CPU. perf top on the TLS terminator during the incident. If OpenSSL symbols dominate user time — the P-256 or RSA montgomery routines — the box is spending itself on handshakes, not on your application.
• Compare connection rate against request rate. nstat -az TcpPassiveOpens sampled a few seconds apart gives new connections; your access log gives requests. A connection rate at or above the request rate means almost nothing is being amortised over keep-alive, which is the signature of this attack.
• Count resumptions. Log $ssl_session_reused in nginx (or ssl_fc_is_resumed in HAProxy). Healthy traffic resumes a large share of its sessions; a flood resumes essentially none.
• Read the error log in bulk. Thousands of SSL_do_handshake() failed or while SSL handshaking lines mean handshakes are being started and abandoned at a rate no browser population would produce.
• Know your ceiling before you need it. openssl s_time -connect host:443 -new measures your full-handshake rate and -reuse measures the resumed rate. The gap between those two numbers is precisely the exposure this attack targets, and it is better measured on a quiet Tuesday than during an incident.

If instead your logs are full of plausible-looking requests and your CPU is in application code, you have a request flood rather than a handshake flood, and how to stop a DDoS attack is the page you want.

What to fix on the server

Each of these is worth doing. None of them is the answer.

Serve an ECDSA certificate. Swapping an RSA-2048 signature for ECDSA P-256 is the single largest change available to you — typically an order of magnitude more handshakes per core. Modern stacks let you configure both and serve whichever the client supports, so there is no compatibility argument left for RSA-only in 2026.

Trim the menu. Offer X25519 and P-256 and stop there. Remove large finite-field DH groups and any remaining RSA key-exchange suites. You cannot stop the attacker choosing the most expensive option you support; you can shorten the list.

Make resumption work for real visitors. Session tickets with sensible rotation, and the TLS 1.3 PSK path, mean your genuine traffic rarely pays for a full handshake. This does nothing to the attack and a great deal to your headroom while it is happening.

Disable client-initiated renegotiation, and prefer TLS 1.3. TLS 1.3 removed renegotiation entirely, which closes the THC-SSL-DOS shape outright. If you still terminate TLS 1.2 anywhere, confirm that client-initiated renegotiation is refused rather than assuming your distribution did it.

Shorten the handshake timeout. nginx's ssl_handshake_timeout defaults to 60 seconds, which is an invitation to hold a worker slot for a minute at no cost. A few seconds is plenty for a real browser.

Keep keep-alive on, and generous. A real visitor's handshake should pay for dozens of requests. Every connection you force a browser to re-establish is free capacity you hand to the attack.

Cap concurrent connections per source, with limit_conn or equivalent. This works against one noisy host and does almost nothing against three thousand residential addresses, for the reasons set out in what is a botnet and rate limiting.

Together these might multiply your survivable handshake rate by ten. The attacker multiplies theirs by renting a second botnet for the price of a lunch, and — this is the part no tuning touches — every one of your defences still runs after the packets have arrived on your uplink and after your CPU has looked at them. A cost you pay per attack packet is a cost the attacker gets to set.

Where a TLS handshake flood is actually stopped

Upstream of the thing holding the private key, cheapest decision first:

1. TCP terminated away from your origin. A source that cannot complete a TCP handshake never reaches the TLS layer at all, which removes every spoofed packet from the problem before any cryptography is considered.
2. Per-source connection-rate admission control. A cap on how fast a single source may start new handshakes, applied before the signature is performed rather than after — with visitors who have already proven themselves admitted ahead of the cap, so a flood cannot crowd out your regulars.
3. Kernel-level drops for proven attackers. Once a source has been judged, its packets should cost a table lookup and nothing else. The whole game is the ratio between what a decision costs you and what it costs them.
4. Aggregate handshake capacity. Many cores across several points of presence have a vastly higher combined crypto ceiling than one origin, and the flood divides across them instead of concentrating on you.
5. Resumption and connection pooling at the edge. When real visitors resume at the nearest point of presence and the edge reaches your origin over a small pool of long-lived connections, your origin's handshake rate stops being a function of your visitor count — or of your attacker's.
6. Only then, request-level defence. Challenges, behavioural signatures and rate limits all presuppose a request. They are the right tools for a Layer 7 flood and they arrive too late for a handshake that is abandoned before any request exists.

One prerequisite sits outside that list and silently voids all of it: your origin must be unreachable except through the protection network. If an attacker can still open a TCP connection to your server's real address, none of the above is in the path — see how to hide your origin IP address. The general shape of this argument, for every attack family rather than this one, is in what is DDoS mitigation.

How Itnetic stops TLS handshake floods

Itnetic's edge is designed around exactly the cost ordering above — push every decision as low as it will go, and let only the traffic that survives all of it be interesting.

Two commercial details matter as much as the packet path. Mitigated attack traffic is never metered against your bandwidth quota — including the certificate chains a handshake flood makes an edge transmit — so a week-long flood does not arrive as an invoice. And going live is two DNS records at your existing provider: no nameserver move, about five minutes, one hostname at a time if you want to watch it first. Itnetic's DDoS protection covers the rest of the pipeline, and every feature named above is on every plan, free included.

A seven-point checklist

1. Measure your own ceiling: openssl s_time -connect yourhost:443 -new versus -reuse, and openssl speed on the TLS terminator.
2. Plot new connections per second next to bandwidth and requests, and alert on it. You cannot respond to a metric you do not graph.
3. Serve an ECDSA certificate, keeping RSA only as a fallback if you still need one.
4. Restrict offered groups and ciphersuites to cheap modern ones; delete large FFDHE groups and RSA key exchange.
5. Verify session resumption is actually happening for real visitors by logging it, not by assuming it.
6. Lower ssl_handshake_timeout to single-digit seconds and confirm client-initiated renegotiation is refused on any TLS 1.2 listener.
7. Firewall the origin to your protection network, so the handshake an attacker pays for is never one your server could have answered.

If this is happening right now, work through how to stop a DDoS attack rather than finishing this page. When it is over, how to test your DDoS protection covers how to confirm — safely and legally — that the next flood meets a filter instead of your private key.