{"record":{"id":"4fa48ebcf1a60b45","repo":"gohugoio/hugo","slug":"error-reading-blob-header-n","errorCode":null,"errorMessage":"Error reading blob header\\n","messagePattern":"Error reading blob header\\\\n","errorType":"console","errorClass":null,"httpStatus":null,"severity":"error","filePath":"internal/warpc/genavif/avif.c","lineNumber":319,"sourceCode":"        uint32_t blob_size = 0;\n\n        // Remove newline character if present\n        line[strcspn(line, \"\\n\")] = 0;\n\n        if (strlen(line) == 0)\n        {\n            continue;\n        }\n\n        input = parse_input_message(line);\n\n        // Next in stream is a blob header defined in https://github.com/bep/textandbinaryreader\n        // T', 'A', 'K', '3', '5', 'E', 'M', '1' id uint32, size uint32\n        uint8_t blob_header[16];\n        size_t read_bytes = fread(blob_header, 1, sizeof(blob_header), stream);\n        if (read_bytes != sizeof(blob_header))\n        {\n            fprintf(stderr, \"Error reading blob header\\n\");\n            goto cleanup;\n        }\n        uint32_t blob_id = *(uint32_t *)&blob_header[8];\n        blob_size = *(uint32_t *)&blob_header[12];\n        blob_data = malloc((size_t)blob_size);\n        if (blob_data == NULL)\n        {\n            // Out of memory. Drain the blob from the input stream so the next\n            // command stays aligned, then report the error to the client instead\n            // of leaving the stream corrupted with no response.\n            drain_bytes(stream, (size_t)blob_size);\n            OutputMessage err_output = {0};\n            err_output.header = input.header;\n            snprintf(err_output.header.err, sizeof(err_output.header.err),\n                     \"out of memory allocating %u bytes for blob data\", blob_size);\n            write_output_message(&err_output);\n            goto cleanup;\n        }","sourceCodeStart":301,"sourceCodeEnd":337,"githubUrl":"https://github.com/gohugoio/hugo/blob/52c9bd7908b4d02d4d0ff8f82a888834d6ee10d2/internal/warpc/genavif/avif.c#L301-L337","documentation":"Printed by genavif's handle_commands (avif.c:319) when fread cannot read the full 16-byte blob header (magic 'TAK35EM1' + uint32 id + uint32 size) that must follow each JSON command line. A short or zero read means the stdin stream ended early, was closed, or is out of sync because a previous command consumed the wrong number of bytes. The worker jumps to cleanup, abandoning this iteration without writing a response.","triggerScenarios":"EOF reached on stdin (host closed the pipe after the JSON line but before sending the blob), a previous iteration left the stream misaligned so what should be the next blob header is actually mid-pixel-data, or the host crashed / was killed between writing the JSON line and the blob header. Reading from a closed file or a truncated test fixture also triggers it.","commonSituations":"Host process killed by context cancellation mid-request, broken pipe silently swallowed by the Go writer, test fixtures that send a JSON line with no following blob, or a prior 'Error reading blob data' that left the stream pointer in the wrong place so the next iteration reads pixel bytes as the magic header.","solutions":["Treat this on the Go side as a fatal worker state: tear down the genavif subprocess and spawn a fresh one rather than reusing the desynchronized pipe.","Confirm the host writes JSON line + 16-byte header + blob payload atomically per command and that no short writes are masked by retries that drop bytes.","Audit the prior command's blob_size field — a wrong size there desyncs every subsequent read.","In tests, always close stdin only after the full blob has been written.","Add a read-side timeout in the Go warpc client so a stuck worker is detected instead of looking like a clean EOF."],"exampleFix":"// before (Go host): wrote JSON then context cancelled before blob\nw.Write(jsonLine)\nblob, err := readPixels(ctx)\nif err != nil { return err } // blob never sent, worker hits short header read next loop\nw.Write(blob)\n\n// after: write JSON + header + blob as one buffered unit, or skip the command entirely on error\nif err != nil { return err }\nw.Write(fullFrame) // jsonLine + 16-byte header + payload, written together","handlingStrategy":"retry","validationCode":"// Write the full request (JSON line + 16-byte header + blob) atomically; if any\n// step fails, discard the worker and start a new one.\nfunc writeRequest(w io.Writer, cmd []byte, blobID uint32, blob []byte) error {\n    if !bytes.HasSuffix(cmd, []byte{'\\n'}) { cmd = append(cmd, '\\n') }\n    hdr := make([]byte, 16)\n    copy(hdr[0:8], []byte(\"TAK35EM1\"))\n    binary.LittleEndian.PutUint32(hdr[8:12], blobID)\n    binary.LittleEndian.PutUint32(hdr[12:16], uint32(len(blob)))\n    frame := append(append(cmd, hdr...), blob...)\n    n, err := w.Write(frame)\n    if err != nil || n != len(frame) { return errShortWrite }\n    return nil\n}","typeGuard":null,"tryCatchPattern":null,"preventionTips":["Build the entire request in memory and write it in one io.Writer.Write call.","Set a write deadline / context on the worker pipe so a stuck host is detected instead of producing a clean EOF.","Treat a short-header read as a fatal worker condition and respawn.","In tests, close stdin only after the full blob payload has been written."],"tags":["rpc","io","protocol","avif","stdin","stream-desync"],"backgroundTag":null,"analyzedSha":"52c9bd7908b4d02d4d0ff8f82a888834d6ee10d2","analyzedAt":"2026-08-09T21:49:36.660Z","contentChangedAt":null,"schemaVersion":2},"datasetVersion":"2026-09-14T05:17:10.506Z"}