Forge: Denial of service in TLS record processing via call stack exhaustion

Summary

Connections created with forge.tls.createConnection() processed received TLS records recursively, one stack level per record. A peer can send one chunk of thousands of small records, such as heartbeat records, which are accepted before the handshake, and process() throws RangeError: Maximum call stack size exceeded, which terminates Node.js processes that use the documented socket wiring. Fixed in node-forge 1.4.1.

Details

In lib/tls.js, c.process() reads one record and passes it to _update(), which calls the handler for the record type from ctTable. Every handler (tls.handleHeartbeat, tls.handleApplicationData, tls.handleHandshake for fragmented messages, tls.handleHelloRequest and the other handshake handlers) ends by calling c.process() again to continue with the next buffered record, or the next message in the same record. Each record therefore adds process → _update → handler to the call stack. In Node.js 24, about 2,300 eight-byte records (under 20 KB) in one chunk are enough for process() to throw RangeError: Maximum call stack size exceeded.

ctTable maps heartbeat records to tls.handleHeartbeat in every client and server state, so heartbeat records are processed before the handshake without any extension negotiation. The attack needs no authentication and no valid handshake messages. Both server connections (attacked by a client) and client connections (attacked by the server they connect to) are affected. Other record types recurse the same way: a handshake message fragmented over many records, many HelloRequest messages in one record, or many application data records after the handshake.

node-forge reports TLS protocol errors through the error callback, so applications do not normally wrap process() in try/catch. When process() is called from a Node.js socket data handler, as in the documented usage, the exception is uncaught and terminates the process. If the exception is caught, the connection is left partially processed and should be discarded.

No specification is violated; this is an implementation flaw (uncontrolled recursion).

The fix makes c.process() iterative. The record-reading code moves into _processRecord(), and c.process() runs it in a loop. A handler's call to c.process() without new data, made while records are being processed, now only flags that the loop should continue. Calls with new data are processed right away as before, so callback order and return values are unchanged.

Patches

Fixed in node-forge 1.4.1. Records are now processed in a loop, so the stack depth no longer depends on the number of records.

Workarounds

Wrap every call to connection.process() in try/catch, and on an exception close the connection (connection.close()) and destroy the underlying socket. This prevents the process crash but not the loss of the attacked connection.

Credits

Reported jointly by Ziyue Wang, Chenchen, Strick Sheng, Liyi Zhou, and Maurice Ng.

Proof of concept

Input: one chunk of N identical heartbeat records, each 8 bytes:

18 03 01 00 03 01 00 00 (content type 24 = heartbeat, TLS 1.0, length 3, heartbeat_request with a 0-byte payload)

// c = forge.tls.createConnection({server: true, ...}) var rec = forge.util.hexToBytes('1803010003010000'); c.process(new Array(5001).join(rec)); // 5,000 records, 40,000 bytes

On 1.4.0 (Node.js 24), process() throws RangeError: Maximum call stack size exceeded after about 2,300 records, before any handshake. The same happens for a client connection. In a Node.js server that calls c.process(data.toString('binary')) from socket.on('data', ...), a client sending these 40,000 bytes terminates the server process. On 1.4.1 all records are processed and the server keeps running.

Regression tests in tests/unit/tls.js, "record processing": 50,000 heartbeat records in one call in server and client mode, a handshake message fragmented over 50,000 records, many HelloRequest messages per record, and checks that byte-by-byte input and exceptions from callbacks behave as before.

Impact

Applications that use node-forge's TLS implementation: connections from forge.tls.createConnection(), including those created by forge.tls.wrapSocket() and the forge.http / forge.xhr TLS connections, which call process() on received data. Any peer that can connect to a forge TLS server, or any server a forge TLS client connects to, can send a small chunk of records (under 20 KB) without authentication. In Node.js applications using the documented usage, the uncaught exception terminates the process, which drops every connection it serves; repeated chunks keep it down. In browsers the impact is limited to the attacked connection. There is no confidentiality or integrity impact.