apache/iceberg · error · BadHeaderException

Not enough header bytes

Error message

Not enough header bytes

What it means

IcebergDecoder.decode reads a fixed header (magic bytes plus fingerprint) before dispatching to the per-fingerprint decoder. If the stream ends before the full header (HEADER bytes plus fingerprint) can be read, it throws BadHeaderException("Not enough header bytes"). This usually means the stream is empty or truncated.

Source

Thrown at core/src/main/java/org/apache/iceberg/data/avro/IcebergDecoder.java:132

    }

    if (resolver != null) {
      Schema writeSchema = resolver.findByFingerprint(fp);
      if (writeSchema != null) {
        addSchema(writeSchema);
        return decoders.get(fp);
      }
    }

    throw new MissingSchemaException("Cannot resolve schema for fingerprint: " + fp);
  }

  @Override
  public D decode(InputStream stream, D reuse) throws IOException {
    byte[] header = HEADER_BUFFER.get();
    try {
      if (!readFully(stream, header)) {
        throw new BadHeaderException("Not enough header bytes");
      }
    } catch (IOException e) {
      throw new IOException("Failed to read header and fingerprint bytes", e);
    }

    if (IcebergEncoder.V1_HEADER[0] != header[0] || IcebergEncoder.V1_HEADER[1] != header[1]) {
      throw new BadHeaderException(
          String.format(
              Locale.ROOT, "Unrecognized header bytes: 0x%02X 0x%02X", header[0], header[1]));
    }

    RawDecoder<D> decoder = getDecoder(FP_BUFFER.get().getLong(2));

    try {
      return decoder.decode(stream, reuse);
    } catch (UncheckedIOException e) {
      throw new AvroRuntimeException(e);
    }

View on GitHub (pinned to 86d9c8fc54)

Solutions

1. Check the stream is non-empty and correctly positioned before decoding (stream.available() > 0 or file length check).
2. Verify the file/stream actually contains Iceberg-encoded records (writer completed and flushed).
3. Reset/rewind the InputStream if it was previously read.
4. Handle BadHeaderException at the call site to treat empty input as an empty batch.

Example fix

// before
decoder.decode(new FileInputStream(emptyFile), null); // BadHeaderException
// after
File f = new File(path);
if (f.length() > 0) {
  decoder.decode(new FileInputStream(f), null);
}

if (file.length() == 0) return; // or treat as empty batch
count bytes available before decode

null

try { return decoder.decode(stream, reuse); } catch (BadHeaderException e) { LOG.warn("Empty or truncated stream"); return null; }

When it happens

Trigger: Calling decode(stream, reuse) with an empty InputStream, or a stream shorter than the header+fingerprint size (e.g. a zero-byte file or a stream already fully consumed).

Common situations: Empty output files from failed writes; re-reading a stream after it was drained; passing wrong stream (e.g. an empty in-memory BAOS) to the decoder.

Understand the failure class

Background: "failed to read file", EACCES, ENOENT and "could not read <path>" errors: when a program can't read a file from disk — this error's family across 49 libraries.

Related errors

• Failed to read header and fingerprint bytes
• Failed to get stream length
• Failed to get stream length: no open stream
• Cannot resolve schema for fingerprint:
• Unrecognized header bytes: 0x%02X 0x%02X

AI-assisted analysis of apache/iceberg@86d9c8fc54 (2026-09-12). Data as JSON: /api/errors/695be0d45dee3ad1. Report an issue: GitHub.