{"record":{"id":"5a326ce7de3e31d4","repo":"prestodb/presto","slug":"the-chunklength-s-must-not-be-negative-or-great","errorCode":null,"errorMessage":"The chunkLength (%s) must not be negative or greater than remaining size (%s)","messagePattern":"The chunkLength \\((.+?)\\) must not be negative or greater than remaining size \\((.+?)\\)","errorType":"error_code","errorClass":"OrcCorruptionException","httpStatus":null,"severity":"error","filePath":"presto-orc/src/main/java/com/facebook/presto/orc/stream/OrcInputStream.java","lineNumber":467,"sourceCode":"            buffer = null;\n            position = 0;\n            length = 0;\n            uncompressedOffset = 0;\n            memoryUsage.setBytes(getRetainedSizeInBytes());\n            return;\n        }\n\n        // 3 byte header\n        // NOTE: this must match BLOCK_HEADER_SIZE\n        currentCompressedBlockOffset = toIntExact(compressedSliceInput.position());\n        int b0 = compressedSliceInput.readUnsignedByte();\n        int b1 = compressedSliceInput.readUnsignedByte();\n        int b2 = compressedSliceInput.readUnsignedByte();\n\n        boolean isUncompressed = (b0 & 0x01) == 1;\n        int chunkLength = (b2 << 15) | (b1 << 7) | (b0 >>> 1);\n        if (chunkLength < 0 || chunkLength > compressedSliceInput.remaining()) {\n            throw new OrcCorruptionException(orcDataSourceId, \"The chunkLength (%s) must not be negative or greater than remaining size (%s)\", chunkLength, compressedSliceInput.remaining());\n        }\n\n        if (isUncompressed) {\n            buffer = ensureCapacity(buffer, chunkLength);\n            length = compressedSliceInput.read(buffer, 0, chunkLength);\n            if (dwrfDecryptor.isPresent()) {\n                buffer = dwrfDecryptor.get().decrypt(buffer, 0, chunkLength);\n                length = buffer.length;\n            }\n            position = 0;\n        }\n        else {\n            sharedDecompressionBuffer.ensureCapacity(chunkLength);\n            byte[] compressedBuffer = sharedDecompressionBuffer.get();\n            int readCompressed = compressedSliceInput.read(compressedBuffer, 0, chunkLength);\n            if (dwrfDecryptor.isPresent()) {\n                compressedBuffer = dwrfDecryptor.get().decrypt(compressedBuffer, 0, chunkLength);\n                readCompressed = compressedBuffer.length;","sourceCodeStart":449,"sourceCodeEnd":485,"githubUrl":"https://github.com/prestodb/presto/blob/55bb57d202de3b926896fa966c2c4a44c779634e/presto-orc/src/main/java/com/facebook/presto/orc/stream/OrcInputStream.java#L449-L485","documentation":"When reading a compressed ORC chunk header, advance() decodes a 3-byte chunkLength; if it is negative or exceeds the bytes remaining in the current compressed block, the header is invalid and OrcCorruptionException is thrown. ORC chunk headers cannot describe more data than the block contains.","triggerScenarios":"advance() (reached from read, seekToCheckpoint, readVarint, skipVarintsInBuffer) parses a chunk header whose decoded chunkLength is >0x7FFF or larger than the remaining bytes of the compressed block — i.e., byte corruption or misaligned block offset.","commonSituations":"Bit-flip/data corruption in the file; seeking to a wrong block offset (bad checkpoint/stale metadata); reading an ORC file with a header compression setting mismatch.","solutions":["Validate file integrity (checksums, orc-tools scan) — this usually indicates real corruption.","Reopen the file with fresh, correctly computed stripe offsets.","Verify the ORC writer's compression kind and the file wasn't transcoded.","Re-copy the damaged segment from source."],"exampleFix":"// before: trusting stale split offsets\n// after: recompute offsets from the current file footer\nOrcMetadata metadata = OrcFileMetadata.read(orcDataSource);\nStripeInfo stripe = metadata.stripeAt(splitStart);\n// ensures block offsets match the actual file bytes","handlingStrategy":"validation","validationCode":"int remaining = currentBlockLength - blockOffset;\nif (decodedChunkLength < 0 || decodedChunkLength > remaining) {\n    throw new IOException(\"corrupt chunk header: \" + decodedChunkLength);\n}","typeGuard":null,"tryCatchPattern":"try { stream.read(...); } catch (OrcCorruptionException e) {\n    // mark file/stripe as corrupt and retry with a fresh read\n    reopenDataSource();\n    throw new DataReadException(\"invalid ORC chunk header\", e);\n}","preventionTips":["Keep stripe/block offsets derived from the same file version being read.","Checksum files after transfer.","Avoid concurrent truncation/overwrite while readers are open.","Scan with orc-tools to catch corruption early."],"tags":["orc","corruption","compression","chunk-header"],"backgroundTag":"invalid-chunk-length","analyzedSha":"55bb57d202de3b926896fa966c2c4a44c779634e","analyzedAt":"2026-09-04T12:50:26.162Z","contentChangedAt":"2026-09-04T12:50:26.162Z","schemaVersion":2},"datasetVersion":"2026-09-11T21:17:09.523Z"}