prestodb/presto · error · OrcCorruptionException

The chunkLength (%s) must not be negative or greater than re

Error message

The chunkLength (%s) must not be negative or greater than remaining size (%s)

What it means

When reading a compressed ORC chunk header, advance() decodes a 3-byte chunkLength; if it is negative or exceeds the bytes remaining in the current compressed block, the header is invalid and OrcCorruptionException is thrown. ORC chunk headers cannot describe more data than the block contains.

Source

Thrown at presto-orc/src/main/java/com/facebook/presto/orc/stream/OrcInputStream.java:467

            buffer = null;
            position = 0;
            length = 0;
            uncompressedOffset = 0;
            memoryUsage.setBytes(getRetainedSizeInBytes());
            return;
        }

        // 3 byte header
        // NOTE: this must match BLOCK_HEADER_SIZE
        currentCompressedBlockOffset = toIntExact(compressedSliceInput.position());
        int b0 = compressedSliceInput.readUnsignedByte();
        int b1 = compressedSliceInput.readUnsignedByte();
        int b2 = compressedSliceInput.readUnsignedByte();

        boolean isUncompressed = (b0 & 0x01) == 1;
        int chunkLength = (b2 << 15) | (b1 << 7) | (b0 >>> 1);
        if (chunkLength < 0 || chunkLength > compressedSliceInput.remaining()) {
            throw new OrcCorruptionException(orcDataSourceId, "The chunkLength (%s) must not be negative or greater than remaining size (%s)", chunkLength, compressedSliceInput.remaining());
        }

        if (isUncompressed) {
            buffer = ensureCapacity(buffer, chunkLength);
            length = compressedSliceInput.read(buffer, 0, chunkLength);
            if (dwrfDecryptor.isPresent()) {
                buffer = dwrfDecryptor.get().decrypt(buffer, 0, chunkLength);
                length = buffer.length;
            }
            position = 0;
        }
        else {
            sharedDecompressionBuffer.ensureCapacity(chunkLength);
            byte[] compressedBuffer = sharedDecompressionBuffer.get();
            int readCompressed = compressedSliceInput.read(compressedBuffer, 0, chunkLength);
            if (dwrfDecryptor.isPresent()) {
                compressedBuffer = dwrfDecryptor.get().decrypt(compressedBuffer, 0, chunkLength);
                readCompressed = compressedBuffer.length;

View on GitHub (pinned to 55bb57d202)

Solutions

1. Validate file integrity (checksums, orc-tools scan) — this usually indicates real corruption.
2. Reopen the file with fresh, correctly computed stripe offsets.
3. Verify the ORC writer's compression kind and the file wasn't transcoded.
4. Re-copy the damaged segment from source.

Example fix

// before: trusting stale split offsets
// after: recompute offsets from the current file footer
OrcMetadata metadata = OrcFileMetadata.read(orcDataSource);
StripeInfo stripe = metadata.stripeAt(splitStart);
// ensures block offsets match the actual file bytes

int remaining = currentBlockLength - blockOffset;
if (decodedChunkLength < 0 || decodedChunkLength > remaining) {
    throw new IOException("corrupt chunk header: " + decodedChunkLength);
}

try { stream.read(...); } catch (OrcCorruptionException e) {
    // mark file/stripe as corrupt and retry with a fresh read
    reopenDataSource();
    throw new DataReadException("invalid ORC chunk header", e);
}

When it happens

Trigger: advance() (reached from read, seekToCheckpoint, readVarint, skipVarintsInBuffer) parses a chunk header whose decoded chunkLength is >0x7FFF or larger than the remaining bytes of the compressed block — i.e., byte corruption or misaligned block offset.

Common situations: Bit-flip/data corruption in the file; seeking to a wrong block offset (bad checkpoint/stale metadata); reading an ORC file with a header compression setting mismatch.

Related errors

• Reset stream has a block offset but stream is not compressed
• HIVE_CURSOR_ERROR
• NOT_SUPPORTED
• Unsupported compression for verification:
• Write-side compression verification failed: %s (uncompressed

AI-assisted analysis of prestodb/presto@55bb57d202 (2026-09-04). Data as JSON: /api/errors/5a326ce7de3e31d4. Report an issue: GitHub.