{"record":{"id":"f6b9b2279c65b699","repo":"prestodb/presto","slug":"zstd-jni-decompressor-failed-with-f6b9b2","errorCode":null,"errorMessage":"Zstd JNI decompressor failed with ","messagePattern":"Zstd JNI decompressor failed with ","errorType":"exception","errorClass":"RuntimeException","httpStatus":null,"severity":"error","filePath":"presto-spark-base/src/main/java/com/facebook/presto/spark/util/PrestoSparkUtils.java","lineNumber":196,"sourceCode":"\n                int written = compress(input.array(), inputOffset, input.remaining(), output.array(), outputOffset, output.remaining());\n                ((Buffer) output).position(output.position() + written);\n            }\n        };\n    }\n\n    private static PageDecompressor createPageDecompressor()\n    {\n        // based on ZstdJniDecompressor\n        return new PageDecompressor()\n        {\n            @Override\n            public int decompress(byte[] input, int inputOffset, int inputLength, byte[] output, int outputOffset, int maxOutputLength)\n            {\n                long size = Zstd.decompressByteArray(output, 0, maxOutputLength, input, inputOffset, inputLength);\n                if (Zstd.isError(size)) {\n                    String errorName = Zstd.getErrorName(size);\n                    throw new RuntimeException(\"Zstd JNI decompressor failed with \" + errorName);\n                }\n                return toIntExact(size);\n            }\n\n            @Override\n            public void decompress(ByteBuffer input, ByteBuffer output)\n            {\n                if (input.isDirect() || output.isDirect() || !input.hasArray() || !output.hasArray()) {\n                    throw new IllegalArgumentException(\"Non-direct byte buffer backed by byte array required\");\n                }\n                int inputOffset = input.arrayOffset() + input.position();\n                int outputOffset = output.arrayOffset() + output.position();\n\n                int written = decompress(input.array(), inputOffset, input.remaining(), output.array(), outputOffset, output.remaining());\n                ((Buffer) output).position(output.position() + written);\n            }\n        };\n    }","sourceCodeStart":178,"sourceCodeEnd":214,"githubUrl":"https://github.com/prestodb/presto/blob/55bb57d202de3b926896fa966c2c4a44c779634e/presto-spark-base/src/main/java/com/facebook/presto/spark/util/PrestoSparkUtils.java#L178-L214","documentation":"Zstd native decompression is invoked through JNI via Zstd.decompressByteArray; when the native library reports an error code, this is translated into a RuntimeException containing the Zstd error name (e.g. DESTINATION_BUFFER_TOO_SMALL, CORRUPTION_DETECTED).","triggerScenarios":"Calling PrestoSparkUtils decompress(byte[],...) with corrupt, truncated, or non-Zstd input data, or an output buffer smaller than the decompressed size.","commonSituations":"Shuffle/spill data corrupted on disk or truncated writes; decompressing data produced by a different codec or version; under-sized output buffer allocated from a stale/incorrect size header.","solutions":["Verify the input bytes are intact Zstd frames (check for truncation/corruption)","Ensure the output buffer is at least the original uncompressed size","Confirm compression and decompression sides use the same codec and Zstd version","Regenerate the corrupted spill/shuffle data"],"exampleFix":null,"handlingStrategy":"try-catch","validationCode":"// verify input integrity before decompress\nif (inputLength == 0 || output == null || maxOutputLength <= 0) throw new IllegalArgumentException(\"bad zstd input\");","typeGuard":null,"tryCatchPattern":"try { int written = decompress(input, inOff, inLen, output, outOff, maxLen); } catch (RuntimeException e) { if (e.getMessage().startsWith(\"Zstd JNI decompressor failed\")) { /* treat data as corrupted: re-fetch or regenerate */ } else throw e; }","preventionTips":["Store/transfer the uncompressed size alongside compressed data and size output buffers accordingly","Checksum compressed spill/shuffle data and verify before decompression","Pin compatible Zstd/JNI library versions on all nodes"],"tags":["java","zstd","decompression","jni","corruption"],"backgroundTag":"zstd-decompression-failed","analyzedSha":"55bb57d202de3b926896fa966c2c4a44c779634e","analyzedAt":"2026-09-04T12:50:26.162Z","contentChangedAt":"2026-09-04T12:50:26.162Z","schemaVersion":2},"datasetVersion":"2026-09-11T21:17:09.523Z"}