{"record":{"id":"cb95b3bbe9e3eb62","repo":"prestodb/presto","slug":"values-in-column-s-are-too-large-to-process-for","errorCode":null,"errorMessage":"Values in column \"%s\" are too large to process for Presto. Requested to read [%s] bytes, when max allowed is [%s] bytes [%s]","messagePattern":"Values in column \"(.+?)\" are too large to process for Presto\\. Requested to read \\[(.+?)\\] bytes, when max allowed is \\[(.+?)\\] bytes \\[(.+?)\\]","errorType":"exception","errorClass":"GenericInternalException","httpStatus":null,"severity":"error","filePath":"presto-orc/src/main/java/com/facebook/presto/orc/reader/SliceDirectBatchStreamReader.java","lineNumber":182,"sourceCode":"                lengthStream.next(offsetVector, nextBatchSize - nullCount);\n                unpackLengthNulls(offsetVector, isNullVector, nextBatchSize - nullCount);\n            }\n        }\n\n        // Calculate the total length for all entries. Note that the values in the offsetVector are still length values now.\n        long totalLength = 0;\n        for (int i = 0; i < nextBatchSize; i++) {\n            totalLength += offsetVector[i];\n        }\n\n        int currentBatchSize = nextBatchSize;\n        readOffset = 0;\n        nextBatchSize = 0;\n        if (totalLength == 0) {\n            return new VariableWidthBlock(currentBatchSize, EMPTY_SLICE, offsetVector, Optional.ofNullable(isNullVector));\n        }\n        if (totalLength > maxSliceSize) {\n            throw new GenericInternalException(\n                    format(\"Values in column \\\"%s\\\" are too large to process for Presto. Requested to read [%s] bytes, when max allowed is [%s] bytes [%s]\",\n                            streamDescriptor.getFieldName(),\n                            totalLength,\n                            maxSliceSize,\n                            streamDescriptor.getOrcDataSourceId()));\n        }\n        if (dataStream == null) {\n            throw new OrcCorruptionException(streamDescriptor.getOrcDataSourceId(), \"Value is not null but data stream is missing\");\n        }\n\n        // allocate enough space to read\n        byte[] data = new byte[toIntExact(totalLength)];\n        Slice slice = Slices.wrappedBuffer(data);\n\n        if (maxCodePointCount < 0) {\n            // unbounded, simply read all data in on shot\n            dataStream.next(data, 0, data.length);\n            convertLengthVectorToOffsetVector(offsetVector);","sourceCodeStart":164,"sourceCodeEnd":200,"githubUrl":"https://github.com/prestodb/presto/blob/55bb57d202de3b926896fa966c2c4a44c779634e/presto-orc/src/main/java/com/facebook/presto/orc/reader/SliceDirectBatchStreamReader.java#L164-L200","documentation":"Fires in readBlock after summing the per-row lengths of a variable-width column: the row group's total byte count exceeds maxSliceSize, so a single Presto block cannot hold the batch. It is a data-size limit, not corruption — typically one huge (or many large) varchar/varbinary values.","triggerScenarios":"Thrown at presto-orc/src/main/java/com/facebook/presto/orc/reader/SliceDirectBatchStreamReader.java:182 when the library encounters an invalid state.","commonSituations":"See trigger scenarios.","solutions":["Increase the max allowed slice size session/config limit if memory permits","Filter or project out the oversized column; avoid SELECT * on wide data","Split or truncate the oversized values at ingestion, or cast to a narrower representation","Repartition data so row groups contain fewer/smaller values per batch"],"exampleFix":null,"handlingStrategy":"validation","validationCode":null,"typeGuard":null,"tryCatchPattern":null,"preventionTips":[],"tags":[],"backgroundTag":null,"analyzedSha":"55bb57d202de3b926896fa966c2c4a44c779634e","analyzedAt":"2026-09-04T12:50:26.162Z","contentChangedAt":"2026-09-04T12:50:26.162Z","schemaVersion":2},"datasetVersion":"2026-09-11T21:17:09.523Z"}