{"record":{"id":"882f80e50dd7976d","repo":"apache/flink","slug":"offset-is-not-monotonically-ascending-offsets-s","errorCode":null,"errorMessage":"Offset is not monotonically ascending. offsets[%s]=%s, offsets[%s]=%s","messagePattern":"Offset is not monotonically ascending\\. offsets\\[(.+?)\\]=(.+?), offsets\\[(.+?)\\]=(.+?)","errorType":"validation","errorClass":"IllegalArgumentException","httpStatus":null,"severity":"error","filePath":"flink-formats/flink-parquet/src/main/java/org/apache/flink/formats/parquet/utils/NestedPositionUtil.java","lineNumber":157,"sourceCode":"        if (nullValuesCount == 0) {\n            return new CollectionPosition(null, offsetsArray, length, valueCount);\n        }\n        return new CollectionPosition(\n                nullCollectionFlags.toArray(), offsetsArray, length, valueCount);\n    }\n\n    public static boolean isOptionalFieldValueNull(int definitionLevel, int maxDefinitionLevel) {\n        return definitionLevel == maxDefinitionLevel - 1;\n    }\n\n    public static long[] calculateLengthByOffsets(\n            boolean[] collectionIsEmpty, long[] arrayOffsets) {\n        LongArrayList lengthList = new LongArrayList(arrayOffsets.length);\n        for (int i = 0; i < arrayOffsets.length - 1; i++) {\n            long offset = arrayOffsets[i];\n            long length = arrayOffsets[i + 1] - offset;\n            if (length < 0) {\n                throw new IllegalArgumentException(\n                        format(\n                                \"Offset is not monotonically ascending. offsets[%s]=%s, offsets[%s]=%s\",\n                                i, arrayOffsets[i], i + 1, arrayOffsets[i + 1]));\n            }\n            if (collectionIsEmpty[i]) {\n                length = 0;\n            }\n            lengthList.add(length);\n        }\n        return lengthList.toArray();\n    }\n\n    private static int getNextCollectionStartIndex(\n            int[] repetitionLevels, int maxRepetitionLevel, int elementIndex) {\n        do {\n            elementIndex++;\n        } while (hasMoreElements(repetitionLevels, elementIndex)\n                && isNotCollectionBeginningMarker(","sourceCodeStart":139,"sourceCodeEnd":175,"githubUrl":"https://github.com/apache/flink/blob/2f3c205e9266cb30240eb7f4fdab15cad629a70f/flink-formats/flink-parquet/src/main/java/org/apache/flink/formats/parquet/utils/NestedPositionUtil.java#L139-L175","documentation":"NestedPositionUtil.calculateLengthByOffsets derives each collection element's length from consecutive offsets. If arrayOffsets is not monotonically non-decreasing (a later offset is smaller than the previous), the computed length is negative and it throws IllegalArgumentException naming the offending indices.","triggerScenarios":"Reading nested (array/map) parquet columns where the reconstructed offset array from definition/repetition levels is malformed - typically a corrupt file, a reader bug in level processing, or offsets produced by a writer violating the row-data offset contract.","commonSituations":"Corrupted parquet files (truncated network copy, bad HDFS block), files written by non-conforming writers, or version skew in the vectorized reading path for nested types.","solutions":["Validate/re-write the offending parquet file (parquet-tools / spark read of the same file usually confirms corruption)","If the file reads fine elsewhere, update Flink to pick up fixes in the nested-column vectorized reader","Quarantine the bad input split and let the job continue with clean files"],"exampleFix":null,"handlingStrategy":"fallback","validationCode":null,"typeGuard":null,"tryCatchPattern":"try { readSplit(split); } catch (IllegalArgumentException e) { if (e.getMessage().contains(\"monotonically ascending\")) { quarantine(split); /* skip bad file, alert */ } else throw e; }","preventionTips":["Verify file integrity (checksums) after transfers to HDFS/S3","Read suspect files with an independent reader (spark/parquet-tools) to confirm corruption","Isolate per-split reads so one bad file fails only that split"],"tags":["parquet","nested-types","offsets","corruption","reader"],"backgroundTag":null,"analyzedSha":"2f3c205e9266cb30240eb7f4fdab15cad629a70f","analyzedAt":"2026-08-14T08:48:24.518Z","schemaVersion":2},"datasetVersion":"2026-08-14T10:17:34.591Z"}