prestodb/presto · error · OrcCorruptionException

Stream size %s of one of the streams for column %s is larger

Error message

Stream size %s of one of the streams for column %s is larger than supported size %s

What it means

DWRF metadata validation in DwrfMetadataReader.toStream: a stream entry in the stripe footer declares a length exceeding the supported maximum for that stream/column. The %s triple gives stream size, column, and supported limit; indicates a corrupt or writer-inconsistent DWRF stripe footer.

Source

Thrown at presto-orc/src/main/java/com/facebook/presto/orc/metadata/DwrfMetadataReader.java:346

        long cpuStart = THREAD_MX_BEAN.getCurrentThreadCpuTime();
        CodedInputStream input = CodedInputStream.newInstance(inputStream);
        DwrfProto.StripeFooter stripeFooter = DwrfProto.StripeFooter.parseFrom(input);
        runtimeStats.addMetricValue("DwrfReadStripeFooterTimeNanos", RuntimeUnit.NANO, THREAD_MX_BEAN.getCurrentThreadCpuTime() - cpuStart);
        return new StripeFooter(
                toStream(orcDataSourceId, stripeFooter.getStreamsList()),
                toColumnEncoding(types, stripeFooter.getColumnsList()),
                stripeFooter.getEncryptedGroupsList().stream()
                        .map(OrcMetadataReader::byteStringToSlice)
                        .collect(toImmutableList()),
                Optional.ofNullable(emptyToNull(stripeFooter.getWriterTimezone()))
                        .map(timezone -> TimeZone.getTimeZone(ZoneId.of(timezone)).toZoneId()));
    }

    private static Stream toStream(OrcDataSourceId orcDataSourceId, DwrfProto.Stream stream)
    {
        // reader doesn't support streams larger than 2GB
        if (stream.getLength() > Integer.MAX_VALUE) {
            throw new OrcCorruptionException(
                    orcDataSourceId,
                    "Stream size %s of one of the streams for column %s is larger than supported size %s",
                    stream.getLength(),
                    stream.getColumn(),
                    Integer.MAX_VALUE);
        }

        return new Stream(
                stream.getColumn(),
                toStreamKind(stream.getKind()),
                toIntExact(stream.getLength()),
                stream.getUseVInts(),
                stream.getSequence(),
                stream.hasOffset() ? Optional.of(stream.getOffset()) : Optional.empty());
    }

    private static List<Stream> toStream(OrcDataSourceId orcDataSourceId, List<DwrfProto.Stream> streams)
    {

View on GitHub (pinned to 55bb57d202)

Solutions

1. Verify the DWRF file integrity and re-obtain it if corrupted
2. Report as a writer bug if the file was written by Presto/DWRF writer
3. Skip the affected stripe/file in recovery pipelines

When it happens

Trigger: Thrown at presto-orc/src/main/java/com/facebook/presto/orc/metadata/DwrfMetadataReader.java:346 when the library encounters an invalid state.

Common situations: See trigger scenarios.

AI-assisted analysis of prestodb/presto@55bb57d202 (2026-09-04). Data as JSON: /api/errors/80aa5449a08073de. Report an issue: GitHub.