{"record":{"id":"43eef12b4e73724f","repo":"prestodb/presto","slug":"compressed-stream-is-truncated","errorCode":null,"errorMessage":"Compressed stream is truncated","messagePattern":"Compressed stream is truncated","errorType":"exception","errorClass":"RcFileCorruptionException","httpStatus":null,"severity":"error","filePath":"presto-rcfile/src/main/java/com/facebook/presto/rcfile/AircompressorDecompressor.java","lineNumber":42,"sourceCode":"public class AircompressorDecompressor\n        implements RcFileDecompressor\n{\n    private final CompressionCodec codec;\n\n    public AircompressorDecompressor(CompressionCodec codec)\n    {\n        this.codec = requireNonNull(codec, \"codec is null\");\n    }\n\n    @Override\n    public void decompress(Slice compressed, Slice uncompressed)\n            throws RcFileCorruptionException\n    {\n        try (CompressionInputStream decompressorStream = codec.createInputStream(compressed.getInput())) {\n            uncompressed.setBytes(0, decompressorStream, uncompressed.length());\n        }\n        catch (IndexOutOfBoundsException | IOException e) {\n            throw new RcFileCorruptionException(e, \"Compressed stream is truncated\");\n        }\n    }\n\n    @Override\n    public void destroy()\n    {\n    }\n}\n","sourceCodeStart":24,"sourceCodeEnd":51,"githubUrl":"https://github.com/prestodb/presto/blob/55bb57d202de3b926896fa966c2c4a44c779634e/presto-rcfile/src/main/java/com/facebook/presto/rcfile/AircompressorDecompressor.java#L24-L51","documentation":"AircompressorDecompressor.decompress uses the aircompressor codec to inflate an RCFile compressed block. If the decompression stream ends prematurely (IOException or IndexOutOfBoundsException), it throws RcFileCorruptionException 'Compressed stream is truncated'. The on-disk block does not contain the full compressed payload it claims.","triggerScenarios":"Reading an RCFile whose compressed block bytes are cut short — the codec's CompressionInputStream hits EOF before producing the expected uncompressed.length() bytes.","commonSituations":"Files corrupted by failed/truncated writes (job killed mid-write, partial HDFS block); wrong byte offsets/sync markers from a writer bug; copying files while they were still being written.","solutions":["Validate/re-copy the source file and compare checksums against the original.","Identify which job produced the file and re-run it; confirm writes completed.","Check reader/writer version compatibility for the compression codec.","If only some stripes are bad, skip corrupt stripes if your reader supports it."],"exampleFix":null,"handlingStrategy":"try-catch","validationCode":"// validate file completeness before reading (size/sync-marker sanity)\nFileSystem fs = path.getFileSystem(conf);\nlong len = fs.getFileStatus(path).getLen();\nif (len == 0 || len < expectedMinimumBlockSize) throw new IllegalStateException(\"File suspiciously small/truncated: \" + path);","typeGuard":null,"tryCatchPattern":"try {\n  rcFileReader.readBlock(block);\n} catch (RcFileCorruptionException e) {\n  if (String.valueOf(e.getMessage()).contains(\"Compressed stream is truncated\")) {\n    log.error(\"Corrupt RCFile block, skipping or re-fetching source: \" + e.getCause());\n  } else throw e;\n}","preventionTips":["Verify checksums (HDFS does this; check local FS too) before reading files","Never copy files before the writing job completes and closes them","Re-run producing jobs on failure instead of consuming partial output","Keep aircompressor/Hadoop versions consistent between writers and readers"],"tags":["rcfile","compression","data-corruption","truncated"],"backgroundTag":"corrupt-compressed-data","analyzedSha":"55bb57d202de3b926896fa966c2c4a44c779634e","analyzedAt":"2026-09-04T12:50:26.162Z","contentChangedAt":"2026-09-04T12:50:26.162Z","schemaVersion":2},"datasetVersion":"2026-09-11T21:17:09.523Z"}