prestodb/presto · error · GenericInternalException

Values in column "%s" are too large to process for Presto. R

Error message

Values in column "%s" are too large to process for Presto. Requested to read [%s] bytes, when max allowed is [%s] bytes [%s]

What it means

Same maxSliceSize guard as the batch reader, applied in the selective (filter-pushdown) path: the summed lengths of the selected positions exceed the block size limit. Caused by very large varchar/varbinary values surviving the filter, not by corruption.

Source

Thrown at presto-orc/src/main/java/com/facebook/presto/orc/reader/SliceDirectSelectiveStreamReader.java:721

                for (int i = 0; i < totalPositions; i++) {
                    boolean isNotNull = nullCount == 0 || !isNullVector[i];
                    if (i == positions[positionIndex]) {
                        if (isNotNull) {
                            totalLength += lengthVector[lengthIndex];
                            maxLength = Math.max(maxLength, lengthVector[lengthIndex]);
                            lengthIndex++;
                        }
                        positionIndex++;
                    }
                    else if (isNotNull) {
                        lengthIndex++;
                    }
                }
            }

            // TODO Do not throw if outputRequired == false
            if (totalLength > context.getMaxSliceSize()) {
                throw new GenericInternalException(
                        format("Values in column \"%s\" are too large to process for Presto. Requested to read [%s] bytes, when max allowed is [%s] bytes [%s]",
                                context.getStreamDescriptor().getFieldName(),
                                totalLength,
                                context.getMaxSliceSize(),
                                context.getStreamDescriptor().getOrcDataSourceId()));
            }
        }

        if (context.isOutputRequired()) {
            if (presentStream != null && context.isNullsAllowed()) {
                nulls = ensureCapacity(nulls, positionCount);
            }
            dataLength = totalLength;
            data = ensureCapacity(data, totalLength);
            offsets = ensureCapacity(offsets, totalPositions + 1, SMALL, INITIALIZE);
        }
        else {
            if (useBatchMode(positionCount, totalPositions)) {

View on GitHub (pinned to 55bb57d202)

Solutions

1. Raise the max slice size limit if cluster memory allows
2. Add selective predicates that reduce the number/size of surviving wide values
3. Project the wide column out of the query or truncate/cast it
4. Rebalance data so individual row groups are smaller

When it happens

Trigger: Thrown at presto-orc/src/main/java/com/facebook/presto/orc/reader/SliceDirectSelectiveStreamReader.java:721 when the library encounters an invalid state.

Common situations: See trigger scenarios.

AI-assisted analysis of prestodb/presto@55bb57d202 (2026-09-04). Data as JSON: /api/errors/3937f32ff0558075. Report an issue: GitHub.