{"record":{"id":"90e407b905812170","repo":"XX-net/XX-Net","slug":"chunk-header-read-fail-crlf","errorCode":null,"errorMessage":"chunk header read fail crlf","messagePattern":"chunk header read fail crlf","errorType":"console","errorClass":null,"httpStatus":null,"severity":"warning","filePath":"code/default/gae_proxy/local/proxy_handler.py","lineNumber":295,"sourceCode":"            self.close_connection = 1\n\n    def go_DIRECT(self):\n        if not self.url.startswith(b\"https\"):\n            xlog.debug(\"Host:%s Direct redirect to https\", self.host)\n            return self.wfile.write(b'HTTP/1.1 301\\r\\nLocation: %s\\r\\nContent-Length: 0\\r\\n\\r\\n' % self.url.replace(b'http://', b'https://', 1))\n\n        request_headers = dict((k.title(), v) for k, v in list(self.headers.items()))\n        payload = self.read_payload()\n\n        xlog.debug(\"DIRECT %s %s from:%s\", self.command, self.url, self.address_string())\n        if direct_handler.handler(self.command, self.host, self.path, request_headers, payload, self.wfile) != \"ok\":\n            self.close_connection = 1\n\n    def read_payload(self):\n        def get_crlf(rfile):\n            crlf = rfile.readline(2)\n            if crlf != b\"\\r\\n\":\n                xlog.warn(\"chunk header read fail crlf\")\n\n        if self.req_payload is not None:\n            return self.req_payload\n\n        payload = b''\n        if b'Content-Length' in self.headers:\n            try:\n                payload_len = int(self.headers.get(b'Content-Length', 0))\n                #xlog.debug(\"payload_len:%d %s %s\", payload_len, self.command, self.path)\n                payload = self.rfile.read(payload_len)\n            except NetWorkIOError as e:\n                xlog.error('handle_method_urlfetch read payload failed:%s', e)\n                return\n        elif b'Transfer-Encoding' in self.headers:\n            # chunked, used by facebook android client\n            payload = \"\"\n            while True:\n                chunk_size_str = self.rfile.readline(65537)","sourceCodeStart":277,"sourceCodeEnd":313,"githubUrl":"https://github.com/XX-net/XX-Net/blob/cfa5bc17b67676e467f37ec50766127e0ab5f0aa/code/default/gae_proxy/local/proxy_handler.py#L277-L313","documentation":"While parsing a chunked request body, read_payload's helper get_crlf read the 2 bytes that must terminate a chunk with CRLF but got something else. This means the client's chunked encoding is malformed (or the stream is out of sync) and the warning is logged mid-parse.","triggerScenarios":"A client declares Transfer-Encoding: chunked but writes incorrect chunk terminators — e.g. uses bare \\n, misses the trailing CRLF after a chunk, or closes mid-body — so rfile.readline(2) != b'\\r\\n'.","commonSituations":"Hand-rolled HTTP clients with wrong chunk framing; clients aborting mid-upload leaving the parser mid-chunk; upstream proxies re-chunking incorrectly; debug tools sending raw chunked bodies by hand.","solutions":["Validate the client's chunked encoder (each chunk: size-line CRLF, data, CRLF; final 0-chunk CRLF CRLF)","Use a standard HTTP library (requests, httpclient) instead of hand-writing chunked bodies","Send Content-Length instead of chunked for small fixed bodies","Capture the wire traffic (wireshark/tcpdump) to see exactly where CRLF is missing"],"exampleFix":"# before (hand-written, missing trailing CRLF)\nsock.send(b'%x\\r\\n' % len(data) + data)  # no trailing CRLF\n\n# after\nsock.send(b'%x\\r\\n' % len(data) + data + b'\\r\\n')\n# end with b'0\\r\\n\\r\\n'","handlingStrategy":"validation","validationCode":"def valid_chunk_terminator(rfile):\n    return rfile.readline(2) == b'\\r\\n'","typeGuard":null,"tryCatchPattern":"crlf = rfile.readline(2)\nif crlf != b'\\r\\n':\n    xlog.warn('chunk header read fail crlf')\n    self.close_connection = 1  # desynced stream; do not reuse connection","preventionTips":["Use standard HTTP client libraries for chunked bodies","Always terminate each chunk and the final 0-chunk with CRLF CRLF","Close the connection on framing errors to avoid poisoning keep-alive reuse"],"tags":["http-proxy","chunked-encoding","protocol-parsing"],"backgroundTag":"malformed-chunked-encoding","analyzedSha":"cfa5bc17b67676e467f37ec50766127e0ab5f0aa","analyzedAt":"2026-08-27T19:28:28.225Z","schemaVersion":2},"datasetVersion":"2026-08-28T00:17:15.603Z"}