apache/iceberg · error · RuntimeException

[Parquet bug] Stream is bad: incorrect bytes remaining

Error message

[Parquet bug] Stream is bad: incorrect bytes remaining <remaining>

What it means

remainingBuffers() delegates to sliceBuffers(length - position) and assumes the remaining bytes must exist because position/length bookkeeping says so. If sliceBuffers hits EOF anyway, the internal accounting is inconsistent, so it rethrows as a RuntimeException labeled a '[Parquet bug] Stream is bad' with the incorrect remaining count. This is an internal invariant violation, not user error.

Solutions

1. Report the Parquet reader bug with the file and stack trace (this is an internal invariant failure)
2. Ensure no code path reads/advances the stream outside the expected reader lifecycle
3. Don't share one MultiBufferInputStream across concurrent or nested readers
4. Recompute the stream from the original buffers/file if the state went bad (retry the task)

Example fix

// before
// stream state inconsistent -> RuntimeException [Parquet bug]
List<ByteBuffer> bufs = stream.remainingBuffers();
// after
// re-create the stream from source data instead of trusting bad state
try (MultiBufferInputStream fresh = MultiBufferInputStream.of(bufferList)) {
  List<ByteBuffer> bufs = fresh.remainingBuffers();
}

if (stream.position() > stream.getLength()) {
  throw new IllegalStateException("Stream position " + stream.position() + " exceeds length " + stream.getLength());
}

try {
  return stream.remainingBuffers();
} catch (RuntimeException e) {
  if (e.getMessage() != null && e.getMessage().startsWith("[Parquet bug]")) {
    throw new IllegalStateException("Stream bookkeeping broken; re-open input", e);
  }
  throw e;
}

When it happens

Trigger: Calling remainingBuffers() on a stream whose position/length no longer match actual buffer contents — e.g. buffers were consumed externally, or position advanced past what the buffer chain can supply.

Common situations: Bugs in Parquet reader implementations mis-tracking consumed bytes; sharing a MultiBufferInputStream between readers that each advance position; custom InputFile/stream wrappers that return inconsistent buffer counts.

Understand the failure class

Background: "This is a bug, please report it": internal invariant violations, unreachable panics, and SNH errors explained — this error's family across 47 libraries.

Related errors

• OptionWriter should only expect at most one field metric…
• AlwaysFalse is a placeholder only
• AlwaysTrue is a placeholder only
• Avro writer does not support variant types
• Batch reading is not supported in non-vectorized reader

AI-assisted analysis of apache/iceberg@86d9c8fc54 (2026-09-12). Data as JSON: /api/errors/426b9aa559ddd80c. Report an issue: GitHub.

Appendix: source

Thrown at core/src/main/java/org/apache/iceberg/io/MultiBufferInputStream.java:224

      } else if (!nextBuffer()) {
        // there are no more buffers
        throw new EOFException();
      }
    }

    return sliceBuffers;
  }

  @Override
  public List<ByteBuffer> remainingBuffers() {
    if (position >= length) {
      return Collections.emptyList();
    }

    try {
      return sliceBuffers(length - position);
    } catch (EOFException e) {
      throw new RuntimeException(
          "[Parquet bug] Stream is bad: incorrect bytes remaining " + (length - position));
    }
  }

  @Override
  public int read(byte[] bytes, int off, int len) {
    if (len <= 0) {
      if (len < 0) {
        throw new IndexOutOfBoundsException("Read length must be greater than 0: " + len);
      }
      return 0;
    }

    if (current == null) {
      return -1;
    }

    int bytesRead = 0;

View on GitHub (pinned to 86d9c8fc54)