prestodb/presto · error · OrcCompressionVerificationException

Write-side compression verification failed: chunk does not d

Error message

Write-side compression verification failed: chunk does not decode back to the original bytes (uncompressedSize=%s compressedSize=%s decompressedSize=%s)

What it means

After decompressing a freshly written chunk, verifyCompressedChunk compares the decompressed length and content against the original bytes. If they differ, the chunk does not round-trip and OrcCompressionVerificationException is thrown to prevent writing corrupt data.

Source

Thrown at presto-orc/src/main/java/com/facebook/presto/orc/OrcOutputBuffer.java:639

        byte[] decompressed = decompressionBufferPool.checkOut(length);
        try {
            int decompressedLength;
            try {
                decompressedLength = verifyDecompressor.decompress(compressed, 0, compressedLength, decompressed, 0, length);
            }
            catch (RuntimeException e) {
                // Any decode failure (ZstdException, MalformedInputException, ...) means the
                // freshly compressed chunk is not decodable. Chain the cause so the original
                // decoder stack trace is preserved.
                throw new OrcCompressionVerificationException(
                        e,
                        "Write-side compression verification failed: %s (uncompressedSize=%s compressedSize=%s)",
                        e.getMessage(),
                        length,
                        compressedLength);
            }
            if (decompressedLength != length || !Arrays.equals(decompressed, 0, length, original, offset, offset + length)) {
                throw new OrcCompressionVerificationException(
                        "Write-side compression verification failed: chunk does not decode back to the original bytes (uncompressedSize=%s compressedSize=%s decompressedSize=%s)",
                        length,
                        compressedLength,
                        decompressedLength);
            }
        }
        finally {
            decompressionBufferPool.checkIn(decompressed);
        }
    }

    @VisibleForTesting
    int getBufferCapacity()
    {
        return slice.length();
    }
}

View on GitHub (pinned to 55bb57d202)

Solutions

Check the reported sizes to determine whether it is a length or content mismatch
Test with a different compression kind to isolate the buggy codec
Update the codec library (e.g. aircompressor, zstd-jni) to the latest version
Disable write-side compression verification if the codec is trusted and verification is too strict

Defensive patterns

Strategy: try-catch

Try / catch

try { orcWriter.write(page); }
catch (OrcCompressionVerificationException e) {
    if (e.getMessage().contains("does not decode back")) {
        /* switch codec or rewrite with verification disabled after root-cause */
    }
}

Prevention

Regression-test compress/decompress round-trips per codec version
Pin codec library versions
Enable verification in CI to catch codec bugs early

When it happens

Trigger: writeChunkToOutputStream verifies a chunk and decompressedLength != original length, or Arrays.equals fails between the decompressed bytes and the original input region.

Common situations: Buggy or misconfigured compressor/decompressor pairs, native library bugs, byte-range comparison bugs, or concurrent modification of buffers during compression.

Related errors

AI-assisted analysis of prestodb/presto@55bb57d202 (2026-09-04). Data as JSON: /api/errors/b35008fd9ac88c51. Report an issue: GitHub.