gohugoio/hugo · error

Error reading blob header\n

Error message

Error reading blob header\n

What it means

Printed by genwebp's handle_commands (webp.c:571) when fread cannot read the full 16-byte blob header following a JSON command line. Same semantics as the genavif counterpart (822): EOF, closed stdin, or stream desync. The worker jumps to cleanup without writing a response.

Source

Thrown at internal/warpc/genwebp/webp.c:571

        uint32_t blob_size = 0;

        // Remove newline character if present
        line[strcspn(line, "\n")] = 0;

        if (strlen(line) == 0)
        {
            continue;
        }

        input = parse_input_message(line);

        // Next in stream is a blob header defined in https://github.com/bep/textandbinaryreader
        // T', 'A', 'K', '3', '5', 'E', 'M', '1' id uint32, size uint32
        uint8_t blob_header[16];
        size_t read_bytes = fread(blob_header, 1, sizeof(blob_header), stream);
        if (read_bytes != sizeof(blob_header))
        {
            fprintf(stderr, "Error reading blob header\n");
            goto cleanup;
        }
        uint32_t blob_id = *(uint32_t *)&blob_header[8];
        blob_size = *(uint32_t *)&blob_header[12];
        blob_data = malloc((size_t)blob_size);
        if (blob_data == NULL)
        {
            // Out of memory. Drain the blob from the input stream so the next
            // command stays aligned, then report the error to the client instead
            // of leaving the stream corrupted with no response.
            drain_bytes(stream, (size_t)blob_size);
            OutputMessage err_output = {0};
            err_output.header = input.header;
            snprintf(err_output.header.err, sizeof(err_output.header.err),
                     "out of memory allocating %u bytes for blob data", blob_size);
            write_output_message(&err_output);
            goto cleanup;
        }

View on GitHub (pinned to 52c9bd7908)

Solutions

1. Treat as fatal worker state on the Go side: respawn genwebp rather than reuse the desynced pipe.
2. Write JSON + 16-byte header + blob as one logical unit; skip the command entirely if any part cannot be produced.
3. Audit the prior command's declared blob_size; a wrong value desyncs subsequent reads.
4. Add a read timeout in the Go warpc client to detect a stuck worker.
5. In tests, close stdin only after the full blob is written.

Example fix

// before (Go host): wrote JSON then errored before blob
w.Write(jsonLine)
pix, err := readPixels(ctx)
if err != nil { return err } // blob never sent
w.Write(blob)

// after: build the full frame first, write atomically or not at all
if err != nil { return err }
frame := append(append(jsonLine, hdr...), pix...)
w.Write(frame)

func writeWebpRequest(w io.Writer, cmd []byte, blobID uint32, blob []byte) error {
    if !bytes.HasSuffix(cmd, []byte{'\n'}) { cmd = append(cmd, '\n') }
    hdr := make([]byte, 16)
    copy(hdr[0:8], []byte("TAK35EM1"))
    binary.LittleEndian.PutUint32(hdr[8:12], blobID)
    binary.LittleEndian.PutUint32(hdr[12:16], uint32(len(blob)))
    frame := append(append(cmd, hdr...), blob...)
    n, err := w.Write(frame)
    if err != nil || n != len(frame) { return errShortWrite }
    return nil
}

When it happens

Trigger: Host closed the pipe after the JSON line but before the blob, host was killed mid-send, the stream is misaligned from a previous bad read so pixel bytes are interpreted as the next header, or a truncated test fixture.

Common situations: Context cancellation killing the host between the JSON line and the blob, broken pipe masked by the Go writer, a prior 'Error reading blob data' leaving the stream pointer wrong, or fixtures that send a JSON line with no blob.

Related errors

• Error reading blob header\n
• [%d] Error reading blob data (size: %llu read: %zu) \n
• [%d] Error reading blob data (size: %llu read: %zu) \n
• Error parsing JSON line\n
• Error parsing JSON line\n

AI-assisted analysis of gohugoio/hugo@52c9bd7908 (2026-08-09). Data as JSON: /api/errors/07031f48822714c5. Report an issue: GitHub.