{"record":{"id":"73bbf85d4c98bb91","repo":"prestodb/presto","slug":"compressed-stream-is-truncated-73bbf8","errorCode":null,"errorMessage":"Compressed stream is truncated","messagePattern":"Compressed stream is truncated","errorType":"exception","errorClass":"RcFileCorruptionException","httpStatus":null,"severity":"error","filePath":"presto-rcfile/src/main/java/com/facebook/presto/rcfile/HadoopDecompressor.java","lineNumber":50,"sourceCode":"    private boolean destroyed;\n\n    public HadoopDecompressor(CompressionCodec codec)\n    {\n        this.codec = requireNonNull(codec, \"codec is null\");\n        decompressor = CodecPool.getDecompressor(codec);\n    }\n\n    @Override\n    public void decompress(Slice compressed, Slice uncompressed)\n            throws RcFileCorruptionException\n    {\n        checkState(!destroyed, \"Codec has been destroyed\");\n        decompressor.reset();\n        try (CompressionInputStream decompressorStream = codec.createInputStream(compressed.getInput(), decompressor)) {\n            uncompressed.setBytes(0, decompressorStream, uncompressed.length());\n        }\n        catch (IndexOutOfBoundsException | IOException e) {\n            throw new RcFileCorruptionException(e, \"Compressed stream is truncated\");\n        }\n    }\n\n    @Override\n    public void destroy()\n    {\n        if (destroyed) {\n            return;\n        }\n        destroyed = true;\n        CodecPool.returnDecompressor(decompressor);\n    }\n}\n","sourceCodeStart":32,"sourceCodeEnd":64,"githubUrl":"https://github.com/prestodb/presto/blob/55bb57d202de3b926896fa966c2c4a44c779634e/presto-rcfile/src/main/java/com/facebook/presto/rcfile/HadoopDecompressor.java#L32-L64","documentation":"HadoopDecompressor.decompress uses a Hadoop codec's decompressor to inflate an RCFile block. An IOException or IndexOutOfBoundsException during inflation indicates the compressed stream ended before yielding the expected bytes, so RcFileCorruptionException 'Compressed stream is truncated' is thrown. Same symptom as the aircompressor path but via Hadoop codecs.","triggerScenarios":"Reading an RCFile block whose compressed payload is incomplete — codec.createInputStream reaches EOF before filling uncompressed.length() bytes.","commonSituations":"Partially written/corrupted files (failed job, disk full); files written by a buggy or incompatible writer version; Hadoop codec version mismatch changing framing expectations.","solutions":["Verify file integrity with checksums and re-copy/re-generate the file.","Re-run the writing job ensuring it completes and closes the file.","Align reader/writer Hadoop and Presto versions for the codec in use.","Isolate whether corruption affects all files or specific ones (writer/region-specific)."],"exampleFix":null,"handlingStrategy":"try-catch","validationCode":"// sanity-check file size and completeness before decompressing blocks\nlong fileLen = fs.getFileStatus(rcFilePath).getLen();\nif (fileLen <= 0) throw new IllegalStateException(\"RCFile is empty or truncated: \" + rcFilePath);","typeGuard":null,"tryCatchPattern":"try {\n  reader.readBlock(value);\n} catch (RcFileCorruptionException e) {\n  if (String.valueOf(e.getMessage()).contains(\"Compressed stream is truncated\")) {\n    handleCorruptStripe(e); // skip, re-read from source, or fail the scan\n  } else throw e;\n}","preventionTips":["Enable and trust checksum verification on the storage layer","Ensure writers close files cleanly (idempotent, retried jobs)","Keep Hadoop codec versions aligned across writer and reader clusters","Quarantine and re-generate files that fail once instead of retrying endlessly"],"tags":["rcfile","compression","hadoop","data-corruption"],"backgroundTag":"corrupt-compressed-data","analyzedSha":"55bb57d202de3b926896fa966c2c4a44c779634e","analyzedAt":"2026-09-04T12:50:26.162Z","contentChangedAt":"2026-09-04T12:50:26.162Z","schemaVersion":2},"datasetVersion":"2026-09-11T21:17:09.523Z"}