prestodb/presto · error · ParquetDecodingException

Failed to decode.

Error message

Failed to decode.

What it means

AbstractNestedBatchReader.readNext wraps any IOException raised while reading or decoding a nested (ARRAY/MAP/STRUCT) column chunk into a bare ParquetDecodingException with the message 'Failed to decode.' and the IOException as cause. Because it is generic, the real root cause (truncated file, checksum failure, corrupt page, network/IO error) is in the chained exception. It indicates the column chunk could not be decoded at all, not that values were wrong.

Source

Thrown at presto-parquet/src/main/java/com/facebook/presto/parquet/batchreader/AbstractNestedBatchReader.java:121

        readOffset = readOffset + nextBatchSize;
        nextBatchSize = batchSize;
    }

    @Override
    public ColumnChunk readNext(Optional<DateTimeZone> timezone)
    {
        ColumnChunk columnChunk = null;
        try {
            seek();
            if (field.isRequired()) {
                columnChunk = readNestedNoNull(timezone);
            }
            else {
                columnChunk = readNestedWithNull(timezone);
            }
        }
        catch (IOException ex) {
            throw new ParquetDecodingException("Failed to decode.", ex);
        }

        readOffset = 0;
        nextBatchSize = 0;
        return columnChunk;
    }

    @Override
    public long getRetainedSizeInBytes()
    {
        return INSTANCE_SIZE +
                (pageReader == null ? 0 : pageReader.getRetainedSizeInBytes()) +
                (dictionary == null ? 0 : dictionary.getRetainedSizeInBytes()) +
                (repetitionLevelDecoder == null ? 0 : repetitionLevelDecoder.getRetainedSizeInBytes()) +
                (definitionLevelDecoder == null ? 0 : definitionLevelDecoder.getRetainedSizeInBytes()) +
                (valuesDecoder == null ? 0 : valuesDecoder.getRetainedSizeInBytes());
    }

View on GitHub (pinned to 55bb57d202)

Solutions

1. Inspect the caused-by IOException chain for the true root cause (EOF, checksum, connection reset, etc.) and act on it.
2. Re-read/refresh the file: if it is truncated or corrupted, restore it from the source or re-run the writing job.
3. Retry the query if the cause was a transient storage (S3/HDFS) error.
4. Validate the file with a Parquet metadata tool (parquet-tools) to confirm structural corruption before debugging the reader.

// preflight: read the file footer to catch truncation early
ParquetMetadata footer = ParquetFileReader.readFooter(configuration, new Path(path));
long fileLen = fileSystem.getFileStatus(new Path(path)).getLen();
if (fileLen < footer.getBlocks().get(footer.getBlocks().size()-1).getEndingPos()) {
    throw new IllegalStateException("File truncated: " + path);
}

try {
    return readNext(timezone);
} catch (ParquetDecodingException e) {
    Throwable root = e.getCause();
    if (root instanceof IOException && isTransientStorageError((IOException) root)) {
        return retryReadWithBackoff();
    }
    throw e;
}

When it happens

Trigger: readNext() on a nested batch reader when the underlying readNested/readNestedWithNull path throws IOException — e.g. failed page read, corrupt dictionary page, or input stream error.

Common situations: Truncated or corrupt Parquet files (failed write, bad copy/S3 multipart issues); HDFS/S3 transient IO errors mid-scan; files written by incompatible writer versions with malformed pages.

Understand the failure class

• Parsing and encoding errors: unexpected token, malformed input — why parsers reject input and how to find the real culprit.

Related errors

• HUDI_CANNOT_OPEN_SPLIT
• Unsupported Parquet encoding:
• We didn't read correct number of definitionLevels
• PARQUET_IO_READ_ERROR
• PARQUET_IO_READ_ERROR

AI-assisted analysis of prestodb/presto@55bb57d202 (2026-09-04). Data as JSON: /api/errors/4708e4c0efdb7643. Report an issue: GitHub.