gohugoio/hugo · error

[%d] Error reading blob data (size: %llu read: %zu) \n

Error message

[%d]  Error reading blob data (size: %llu read: %zu) \n

What it means

Printed by genavif's handle_commands (avif.c:341) when the blob header declared `size` bytes but fread returned fewer. The message logs blob_id, declared size, and actual bytes read. The worker frees nothing extra and jumps to cleanup, leaving the partially-read bytes consumed. This is the canonical stream-desync signal: the host and worker disagree on payload length.

Source

Thrown at internal/warpc/genavif/avif.c:341

        blob_size = *(uint32_t *)&blob_header[12];
        blob_data = malloc((size_t)blob_size);
        if (blob_data == NULL)
        {
            // Out of memory. Drain the blob from the input stream so the next
            // command stays aligned, then report the error to the client instead
            // of leaving the stream corrupted with no response.
            drain_bytes(stream, (size_t)blob_size);
            OutputMessage err_output = {0};
            err_output.header = input.header;
            snprintf(err_output.header.err, sizeof(err_output.header.err),
                     "out of memory allocating %u bytes for blob data", blob_size);
            write_output_message(&err_output);
            goto cleanup;
        }
        read_bytes = fread(blob_data, 1, (size_t)blob_size, stream);
        if (read_bytes != (size_t)blob_size)
        {
            fprintf(stderr, "[%d]  Error reading blob data (size: %llu read: %zu) \n", blob_id, (unsigned long long)blob_size, read_bytes);
            goto cleanup;
        }

        OutputMessage output = {0};
        output.header = input.header;

        if (strcmp(input.header.command, "decode") == 0)
        {
            avifDecoder *decoder = avifDecoderCreate();
            if (decoder == NULL)
            {
                snprintf(output.header.err, sizeof(output.header.err), "Failed to create AVIF decoder");
                write_output_message(&output);
                goto cleanup;
            }

            decoder->ignoreExif = AVIF_TRUE;
            decoder->ignoreXMP = AVIF_TRUE;

View on GitHub (pinned to 52c9bd7908)

Solutions

1. Compare the declared size against the actual byte count the Go side computes for the payload (width*height*bytesPerPixel*frameCount) and assert they match before writing the header.
2. Recompute size from the same source-of-truth used to allocate the pixel buffer, not from a cached field.
3. Check for partial writes: a Go io.Writer to a closed pipe can return (n<len, nil) on some platforms — treat any n != len as fatal.
4. If the worker is killed and respawned, ensure the new worker is not handed a half-drained pipe.
5. Log the declared size on the Go side at send time and correlate with the size printed here.

Example fix

// before: size computed from cached stride that no longer matches the image
size := uint32(cachedStride * img.Bounds().Dy())
hdr[12:] = size

// after: compute from the actual bytes about to be written
pix := rgbaPixels(img)
size := uint32(len(pix))
binary.LittleEndian.PutUint32(hdr[12:], size)
w.Write(hdr); w.Write(pix)

// Derive the blob size from the exact bytes you will send, not from a cached field.
func blobHeader(id uint32, payload []byte) []byte {
    h := make([]byte, 16)
    copy(h[0:8], []byte("TAK35EM1"))
    binary.LittleEndian.PutUint32(h[8:12], id)
    binary.LittleEndian.PutUint32(h[12:16], uint32(len(payload)))
    return h
}

When it happens

Trigger: The 16-byte header's size field does not match the bytes the host actually writes (off-by-one in stride*width*height, wrong pixel format byte count, endianness confusion when packing the size), the host closes the pipe partway through the blob, or a previous over-read advanced the stream past the true start of the blob.

Common situations: Re-encoding HDR/gain-map content where the Go side computed size with 4 bytes/pixel but the worker expected the size the host declared (mismatch after a depth change), animated AVIF frame counts differing between declared and actual, partial writes from a buffered writer that was flushed early, or test fixtures with a stale size field after the source image was edited.

Related errors

• Error reading blob header\n
• [%d] Error reading blob data (size: %llu read: %zu) \n
• Error reading blob header\n
• Error parsing JSON line\n
• Error: Line did not parse to a valid JSON object\n

AI-assisted analysis of gohugoio/hugo@52c9bd7908 (2026-08-09). Data as JSON: /api/errors/cd86e5a46a9af7ba. Report an issue: GitHub.