prestodb/presto · error · RcFileCorruptionException

Compressed stream is truncated

Error message

Compressed stream is truncated

What it means

HadoopDecompressor.decompress uses a Hadoop codec's decompressor to inflate an RCFile block. An IOException or IndexOutOfBoundsException during inflation indicates the compressed stream ended before yielding the expected bytes, so RcFileCorruptionException 'Compressed stream is truncated' is thrown. Same symptom as the aircompressor path but via Hadoop codecs.

Source

Thrown at presto-rcfile/src/main/java/com/facebook/presto/rcfile/HadoopDecompressor.java:50

    private boolean destroyed;

    public HadoopDecompressor(CompressionCodec codec)
    {
        this.codec = requireNonNull(codec, "codec is null");
        decompressor = CodecPool.getDecompressor(codec);
    }

    @Override
    public void decompress(Slice compressed, Slice uncompressed)
            throws RcFileCorruptionException
    {
        checkState(!destroyed, "Codec has been destroyed");
        decompressor.reset();
        try (CompressionInputStream decompressorStream = codec.createInputStream(compressed.getInput(), decompressor)) {
            uncompressed.setBytes(0, decompressorStream, uncompressed.length());
        }
        catch (IndexOutOfBoundsException | IOException e) {
            throw new RcFileCorruptionException(e, "Compressed stream is truncated");
        }
    }

    @Override
    public void destroy()
    {
        if (destroyed) {
            return;
        }
        destroyed = true;
        CodecPool.returnDecompressor(decompressor);
    }
}

View on GitHub (pinned to 55bb57d202)

Solutions

1. Verify file integrity with checksums and re-copy/re-generate the file.
2. Re-run the writing job ensuring it completes and closes the file.
3. Align reader/writer Hadoop and Presto versions for the codec in use.
4. Isolate whether corruption affects all files or specific ones (writer/region-specific).

// sanity-check file size and completeness before decompressing blocks
long fileLen = fs.getFileStatus(rcFilePath).getLen();
if (fileLen <= 0) throw new IllegalStateException("RCFile is empty or truncated: " + rcFilePath);

try {
  reader.readBlock(value);
} catch (RcFileCorruptionException e) {
  if (String.valueOf(e.getMessage()).contains("Compressed stream is truncated")) {
    handleCorruptStripe(e); // skip, re-read from source, or fail the scan
  } else throw e;
}

When it happens

Trigger: Reading an RCFile block whose compressed payload is incomplete — codec.createInputStream reaches EOF before filling uncompressed.length() bytes.

Common situations: Partially written/corrupted files (failed job, disk full); files written by a buggy or incompatible writer version; Hadoop codec version mismatch changing framing expectations.

Related errors

• Compressed stream is truncated
• Write-side compression verification failed: %s (uncompressed
• Write-side compression verification failed: chunk does not d
• Unknown codec: ${codecName}
• Deserialized MapBlock violates invariants: key %d, value %d

AI-assisted analysis of prestodb/presto@55bb57d202 (2026-09-04). Data as JSON: /api/errors/73bbf85d4c98bb91. Report an issue: GitHub.