prestodb/presto · error · GenericInternalException

Values in column "%s" are too large to process for Presto. R

Error message

Values in column "%s" are too large to process for Presto. Requested to read [%s] bytes, when max allowed is [%s] bytes [%s]

What it means

Fires in readBlock after summing the per-row lengths of a variable-width column: the row group's total byte count exceeds maxSliceSize, so a single Presto block cannot hold the batch. It is a data-size limit, not corruption — typically one huge (or many large) varchar/varbinary values.

Source

Thrown at presto-orc/src/main/java/com/facebook/presto/orc/reader/SliceDirectBatchStreamReader.java:182

                lengthStream.next(offsetVector, nextBatchSize - nullCount);
                unpackLengthNulls(offsetVector, isNullVector, nextBatchSize - nullCount);
            }
        }

        // Calculate the total length for all entries. Note that the values in the offsetVector are still length values now.
        long totalLength = 0;
        for (int i = 0; i < nextBatchSize; i++) {
            totalLength += offsetVector[i];
        }

        int currentBatchSize = nextBatchSize;
        readOffset = 0;
        nextBatchSize = 0;
        if (totalLength == 0) {
            return new VariableWidthBlock(currentBatchSize, EMPTY_SLICE, offsetVector, Optional.ofNullable(isNullVector));
        }
        if (totalLength > maxSliceSize) {
            throw new GenericInternalException(
                    format("Values in column \"%s\" are too large to process for Presto. Requested to read [%s] bytes, when max allowed is [%s] bytes [%s]",
                            streamDescriptor.getFieldName(),
                            totalLength,
                            maxSliceSize,
                            streamDescriptor.getOrcDataSourceId()));
        }
        if (dataStream == null) {
            throw new OrcCorruptionException(streamDescriptor.getOrcDataSourceId(), "Value is not null but data stream is missing");
        }

        // allocate enough space to read
        byte[] data = new byte[toIntExact(totalLength)];
        Slice slice = Slices.wrappedBuffer(data);

        if (maxCodePointCount < 0) {
            // unbounded, simply read all data in on shot
            dataStream.next(data, 0, data.length);
            convertLengthVectorToOffsetVector(offsetVector);

View on GitHub (pinned to 55bb57d202)

Solutions

1. Increase the max allowed slice size session/config limit if memory permits
2. Filter or project out the oversized column; avoid SELECT * on wide data
3. Split or truncate the oversized values at ingestion, or cast to a narrower representation
4. Repartition data so row groups contain fewer/smaller values per batch

When it happens

Trigger: Thrown at presto-orc/src/main/java/com/facebook/presto/orc/reader/SliceDirectBatchStreamReader.java:182 when the library encounters an invalid state.

Common situations: See trigger scenarios.

AI-assisted analysis of prestodb/presto@55bb57d202 (2026-09-04). Data as JSON: /api/errors/cb95b3bbe9e3eb62. Report an issue: GitHub.