{"record":{"id":"f3dd073b2e0701b4","repo":"prestodb/presto","slug":"unexpected-prestosparkmutablerow-buffer-and-ar","errorCode":null,"errorMessage":"Unexpected PrestoSparkMutableRow: 'buffer' and 'array' fields are both null","messagePattern":"Unexpected PrestoSparkMutableRow: 'buffer' and 'array' fields are both null","errorType":"exception","errorClass":"IllegalArgumentException","httpStatus":null,"severity":"error","filePath":"presto-spark-base/src/main/java/com/facebook/presto/spark/execution/shuffle/PrestoSparkShufflePageInput.java","lineNumber":107,"sourceCode":"                        ByteBuffer buffer = row.getBuffer();\n                        verify(buffer.remaining() >= 2, \"row data is expected to be at least 2 bytes long\");\n                        currentIteratorProcessedBytes += buffer.remaining();\n                        short entryRowCount = getShortLittleEndian(buffer);\n                        rowCount += entryRowCount;\n                        currentIteratorProcessedRows += entryRowCount;\n                        ((Buffer) buffer).position(buffer.position() + 2);\n                        output.writeBytes(buffer.array(), buffer.arrayOffset() + buffer.position(), buffer.remaining());\n                    }\n                    else if (row.getArray() != null) {\n                        verify(row.getLength() >= 2, \"row data is expected to be at least 2 bytes long\");\n                        currentIteratorProcessedBytes += row.getLength();\n                        short entryRowCount = getShortLittleEndian(row.getArray(), row.getOffset());\n                        rowCount += entryRowCount;\n                        currentIteratorProcessedRows += entryRowCount;\n                        output.writeBytes(row.getArray(), row.getOffset() + 2, row.getLength() - 2);\n                    }\n                    else {\n                        throw new IllegalArgumentException(\"Unexpected PrestoSparkMutableRow: 'buffer' and 'array' fields are both null\");\n                    }\n                }\n                long end = System.currentTimeMillis();\n                shuffleStats.accumulate(\n                        currentIteratorProcessedRows,\n                        currentIteratorProcessedRowBatches,\n                        currentIteratorProcessedBytes,\n                        end - start);\n                if (!iterator.hasNext()) {\n                    shuffleStatsCollector.add(new PrestoSparkShuffleStats(\n                            input.getFragmentId(),\n                            taskId,\n                            READ,\n                            shuffleStats.getProcessedRows(),\n                            shuffleStats.getProcessedRowBatches(),\n                            shuffleStats.getProcessedBytes(),\n                            shuffleStats.getElapsedWallTimeMills()));\n                    shuffleStats.reset();","sourceCodeStart":89,"sourceCodeEnd":125,"githubUrl":"https://github.com/prestodb/presto/blob/55bb57d202de3b926896fa966c2c4a44c779634e/presto-spark-base/src/main/java/com/facebook/presto/spark/execution/shuffle/PrestoSparkShufflePageInput.java#L89-L125","documentation":"When PrestoSparkShufflePageInput.getNextPage() decodes shuffle rows into a page, each PrestoSparkMutableRow must carry its payload in either the 'buffer' or 'array' field. If a row has both null, the data layout is invalid, and IllegalArgumentException 'Unexpected PrestoSparkMutableRow: buffer and array fields are both null' is thrown rather than producing a corrupt page.","triggerScenarios":"Iterating the shuffle row iterator in getNextPage, a row is neither a buffer-backed nor array-backed row (both fields null) — thrown at PrestoSparkShufflePageInput.java:107 when the writer-side row layout doesn't match the reader's expectations.","commonSituations":"Mixed Presto/Spark shuffle format versions between writer and reader, a corrupted local shuffle file or truncated shuffle data being replayed, wrong shuffle info translator pairing read/write codecs, or task retry reading a partially written shuffle.","solutions":["Verify writer and reader use the same shuffle format/version (matching PrestoSparkLocalShuffleInfoTranslator on both sides)","Check the local shuffle files for corruption or truncation (task retries, disk issues) and re-run the failing stage","Confirm no mixed Presto version executors in the cluster (rolling-upgrade skew) writing incompatible rows","Inspect row.getOffset()/getLength() handling — ensure the writer always sets either buffer or array"],"exampleFix":"// before\n// reading shuffle written by a different Presto version translator\nSparkShuffleWriteInfo = translatorV1.toWriteInfo(...)\n// after\n// use the same translator version on writer and reader\nSparkShuffleWriteInfo = sameTranslatorVersion.toWriteInfo(...)","handlingStrategy":"try-catch","validationCode":"// before consuming shuffle\nif (!row.hasBuffer() && !row.hasArray()) {\n    throw new IllegalStateException(\"Corrupt shuffle row: both buffer and array null\");\n}","typeGuard":"boolean isValidRow(PrestoSparkMutableRow row) {\n    return row.getBuffer() != null || row.getArray() != null;\n}","tryCatchPattern":"try {\n    page = shufflePageInput.getNextPage();\n} catch (IllegalArgumentException e) {\n    if (e.getMessage().contains(\"'buffer' and 'array' fields are both null\")) {\n        // treat as corrupted/mismatched shuffle; retry stage or fail task cleanly\n    }\n}","preventionTips":["Use the same shuffle translator/format version on writer and reader","Avoid mixed Presto versions across executors during rolling upgrades","Verify local shuffle files are not truncated after task retries"],"tags":["shuffle","data-corruption","spark","page-decoding","version-skew"],"backgroundTag":"shuffle-format-mismatch","analyzedSha":"55bb57d202de3b926896fa966c2c4a44c779634e","analyzedAt":"2026-09-04T12:50:26.162Z","contentChangedAt":"2026-09-04T12:50:26.162Z","schemaVersion":2},"datasetVersion":"2026-09-11T21:17:09.523Z"}