What causes "chunk header read fail crlf" in XX-net/XX-Net?

Validate the client's chunked encoder (each chunk: size-line CRLF, data, CRLF; final 0-chunk CRLF CRLF)

chunk header read fail crlf

Error message

chunk header read fail crlf

What it means

While parsing a chunked request body, read_payload's helper get_crlf read the 2 bytes that must terminate a chunk with CRLF but got something else. This means the client's chunked encoding is malformed (or the stream is out of sync) and the warning is logged mid-parse.

Source

Thrown at code/default/gae_proxy/local/proxy_handler.py:295

            self.close_connection = 1

    def go_DIRECT(self):
        if not self.url.startswith(b"https"):
            xlog.debug("Host:%s Direct redirect to https", self.host)
            return self.wfile.write(b'HTTP/1.1 301\r\nLocation: %s\r\nContent-Length: 0\r\n\r\n' % self.url.replace(b'http://', b'https://', 1))

        request_headers = dict((k.title(), v) for k, v in list(self.headers.items()))
        payload = self.read_payload()

        xlog.debug("DIRECT %s %s from:%s", self.command, self.url, self.address_string())
        if direct_handler.handler(self.command, self.host, self.path, request_headers, payload, self.wfile) != "ok":
            self.close_connection = 1

    def read_payload(self):
        def get_crlf(rfile):
            crlf = rfile.readline(2)
            if crlf != b"\r\n":
                xlog.warn("chunk header read fail crlf")

        if self.req_payload is not None:
            return self.req_payload

        payload = b''
        if b'Content-Length' in self.headers:
            try:
                payload_len = int(self.headers.get(b'Content-Length', 0))
                #xlog.debug("payload_len:%d %s %s", payload_len, self.command, self.path)
                payload = self.rfile.read(payload_len)
            except NetWorkIOError as e:
                xlog.error('handle_method_urlfetch read payload failed:%s', e)
                return
        elif b'Transfer-Encoding' in self.headers:
            # chunked, used by facebook android client
            payload = ""
            while True:
                chunk_size_str = self.rfile.readline(65537)

View on GitHub (pinned to cfa5bc17b6)

Solutions

Validate the client's chunked encoder (each chunk: size-line CRLF, data, CRLF; final 0-chunk CRLF CRLF)
Use a standard HTTP library (requests, httpclient) instead of hand-writing chunked bodies
Send Content-Length instead of chunked for small fixed bodies
Capture the wire traffic (wireshark/tcpdump) to see exactly where CRLF is missing

Example fix

# before (hand-written, missing trailing CRLF)
sock.send(b'%x\r\n' % len(data) + data)  # no trailing CRLF

# after
sock.send(b'%x\r\n' % len(data) + data + b'\r\n')
# end with b'0\r\n\r\n'

Defensive patterns

Strategy: validation

Validate before calling

def valid_chunk_terminator(rfile):
    return rfile.readline(2) == b'\r\n'

Try / catch

crlf = rfile.readline(2)
if crlf != b'\r\n':
    xlog.warn('chunk header read fail crlf')
    self.close_connection = 1  # desynced stream; do not reuse connection

Prevention

Use standard HTTP client libraries for chunked bodies
Always terminate each chunk and the final 0-chunk with CRLF CRLF
Close the connection on framing errors to avoid poisoning keep-alive reuse

When it happens

Trigger: A client declares Transfer-Encoding: chunked but writes incorrect chunk terminators — e.g. uses bare \n, misses the trailing CRLF after a chunk, or closes mid-body — so rfile.readline(2) != b'\r\n'.

Common situations: Hand-rolled HTTP clients with wrong chunk framing; clients aborting mid-upload leaving the parser mid-chunk; upstream proxies re-chunking incorrectly; debug tools sending raw chunked bodies by hand.

Related errors

AI-assisted analysis of XX-net/XX-Net@cfa5bc17b6 (2026-08-27). Data as JSON: /api/errors/90e407b905812170. Report an issue: GitHub.