prestodb/presto · error · OrcCorruptionException

Invalid postscript length %s

Error message

Invalid postscript length %s

What it means

The last byte of an ORC/DWRF file encodes the length of the PostScript section. The library throws OrcCorruptionException when that byte value is >= the buffer size actually read, meaning the postscript cannot possibly fit inside the file tail — a sign of corruption or a non-ORC file.

Source

Thrown at presto-orc/src/main/java/com/facebook/presto/orc/cache/StorageOrcFileTailSource.java:82

    }

    @Override
    public OrcFileTail getOrcFileTail(OrcDataSource orcDataSource, MetadataReader metadataReader, Optional<OrcWriteValidation> writeValidation, boolean cacheable, long fileModificationTime)
            throws IOException
    {
        long size = orcDataSource.getSize();
        if (size <= MAGIC.length()) {
            throw new OrcCorruptionException(orcDataSource.getId(), "Invalid file size %s", size);
        }

        // Read the tail of the file
        byte[] buffer = new byte[toIntExact(min(size, expectedFooterSizeInBytes))];
        orcDataSource.readFully(size - buffer.length, buffer);

        // get length of PostScript - last byte of the file
        int postScriptSize = buffer[buffer.length - SIZE_OF_BYTE] & 0xff;
        if (postScriptSize >= buffer.length) {
            throw new OrcCorruptionException(orcDataSource.getId(), "Invalid postscript length %s", postScriptSize);
        }

        // decode the post script
        PostScript postScript;
        try {
            postScript = metadataReader.readPostScript(buffer, buffer.length - SIZE_OF_BYTE - postScriptSize, postScriptSize);
        }
        catch (OrcCorruptionException e) {
            // check if this is an ORC file and not an RCFile or something else
            if (!isValidHeaderMagic(orcDataSource)) {
                throw new OrcCorruptionException(orcDataSource.getId(), "Not an ORC file");
            }
            throw e;
        }

        // verify this is a supported version
        checkOrcVersion(orcDataSource, postScript.getVersion());
        validateWrite(writeValidation, orcDataSource, validation -> validation.getVersion().equals(postScript.getVersion()), "Unexpected version");

View on GitHub (pinned to 55bb57d202)

Solutions

  1. Check file integrity (checksum/size against the writer's manifest) and re-fetch the file from source.
  2. Confirm the file was written completely (look for a _SUCCESS/commit marker in the output directory).
  3. Re-generate the file with the ORC writer; the tail bytes are unrecoverable if corrupt.
  4. Ensure you are not reading a non-ORC file by mistake (check the 'ORC' magic header).

Example fix

// before
byte[] buffer = new byte[toIntExact(min(size, expectedFooterSizeInBytes))];
// after
byte[] buffer = new byte[toIntExact(min(size, Math.max(expectedFooterSizeInBytes, OrcFile.Magic.length() + 1)))];
// and validate size >= 4 before attempting read
Defensive patterns

Strategy: validation

Validate before calling

long size = orcDataSource.getSize();
if (size <= 3 || buffer[buffer.length - 1] >= buffer.length) {
    throw new SkipFileException("Implausible postscript in " + orcDataSource.getId());
}

Try / catch

try { tail = source.getOrcFileTail(ds, reader, validation, false, time); }
catch (OrcCorruptionException e) { quarantine(ds.getId(), e); }

Prevention

When it happens

Trigger: getOrcFileTail reads the last expectedFooterSizeInBytes bytes and the final byte (postScriptSize) is >= buffer.length; typical when the file is tiny, truncated mid-tail, or the 'footer size' byte region is garbage.

Common situations: Files truncated to fewer bytes than the real footer/postscript, files written by broken writers, or accidental reads of random binary/text files as ORC.

Related errors

AI-assisted analysis of prestodb/presto@55bb57d202 (2026-09-04). Data as JSON: /api/errors/08fbd136bbc26ddb. Report an issue: GitHub.