{"record":{"id":"9d5c784d925cb67c","repo":"prestodb/presto","slug":"invalid-offset-s-s","errorCode":null,"errorMessage":"Invalid offset: %s > %s","messagePattern":"Invalid offset: (.+?) > (.+?)","errorType":"exception","errorClass":"IllegalArgumentException","httpStatus":null,"severity":"error","filePath":"presto-parquet/src/main/java/com/facebook/presto/parquet/spark/VariantUtil.java","lineNumber":505,"sourceCode":"    // Throw `MALFORMED_VARIANT` if the variant is malformed. An out-of-bound `id` is also considered\n    // a malformed variant because it is read from the corresponding variant value.\n    public static String getMetadataKey(byte[] metadata, int id)\n    {\n        checkIndex(0, metadata.length);\n        // Extracts the highest 2 bits in the metadata header to determine the integer size of the\n        // offset list.\n        int offsetSize = ((metadata[0] >> 6) & 0x3) + 1;\n        int dictSize = readUnsigned(metadata, 1, offsetSize);\n        if (id >= dictSize) {\n            throw new IllegalArgumentException(String.format(\"Index out of bound: %s (size: %s)\", id, dictSize));\n        }\n        // There are a header byte, a `dictSize` with `offsetSize` bytes, and `(dictSize + 1)` offsets\n        // before the string data.\n        int stringStart = 1 + (dictSize + 2) * offsetSize;\n        int offset = readUnsigned(metadata, 1 + (id + 1) * offsetSize, offsetSize);\n        int nextOffset = readUnsigned(metadata, 1 + (id + 2) * offsetSize, offsetSize);\n        if (offset > nextOffset) {\n            throw new IllegalArgumentException(String.format(\"Invalid offset: %s > %s\", offset, nextOffset));\n        }\n        checkIndex(stringStart + nextOffset - 1, metadata.length);\n        return new String(metadata, stringStart + offset, nextOffset - offset, UTF_8);\n    }\n}\n","sourceCodeStart":487,"sourceCodeEnd":511,"githubUrl":"https://github.com/prestodb/presto/blob/55bb57d202de3b926896fa966c2c4a44c779634e/presto-parquet/src/main/java/com/facebook/presto/parquet/spark/VariantUtil.java#L487-L511","documentation":"When reading a string from the Variant metadata dictionary, the accessor reads the offset and nextOffset for the requested id and requires offset <= nextOffset (offsets must be non-decreasing). This IllegalArgumentException signals the dictionary offsets are inconsistent — the variant metadata bytes are malformed, so the string span [offset, nextOffset) is invalid and the reader refuses to slice it.","triggerScenarios":"Reading dictionary entry `id` from variant metadata where readUnsigned(metadata, 1+(id+1)*offsetSize, offsetSize) > readUnsigned(metadata, 1+(id+2)*offsetSize, offsetSize) — i.e. corrupt or out-of-spec offset table in the metadata byte[].","commonSituations":"Corrupted Parquet variant columns; metadata written by a non-conforming producer; parsing metadata with the wrong offsetSize because the header byte was damaged; truncation that garbles the offset list.","solutions":["Validate that the metadata offset list is non-decreasing before dereferencing entries","Confirm the metadata header byte (offset size bits) is intact and matches the array length","Re-read/regenerate the source data or re-write the variant column with a conforming writer","Catch IllegalArgumentException and treat the row's variant metadata as corrupt"],"exampleFix":"// before\nString s = VariantUtil.getMetadataString(metadata, id);\n// after\ntry {\n    String s = VariantUtil.getMetadataString(metadata, id);\n} catch (IllegalArgumentException e) {\n    // malformed variant metadata: log row and skip\n}","handlingStrategy":"try-catch","validationCode":"// Java\nstatic boolean hasMonotonicOffsets(byte[] metadata, int id, int offsetSize) {\n    int a = VariantUtil.readUnsigned(metadata, 1 + (id + 1) * offsetSize, offsetSize);\n    int b = VariantUtil.readUnsigned(metadata, 1 + (id + 2) * offsetSize, offsetSize);\n    return a <= b;\n}","typeGuard":null,"tryCatchPattern":"// Java\ntry {\n    String s = VariantUtil.getMetadataString(metadata, id);\n} catch (IllegalArgumentException e) {\n    log.warn(\"Corrupt variant metadata offsets\", e);\n    // return null / skip row\n}","preventionTips":["Validate the metadata header byte and total length before parsing the offset list","Treat non-monotonic offsets as corruption and quarantine the row","Re-export corrupted Parquet files with a conforming writer","Monitor this error rate to catch storage or writer corruption early"],"tags":["variant","malformed-data","offset-corruption","parquet"],"backgroundTag":"malformed-variant-metadata-offsets","analyzedSha":"55bb57d202de3b926896fa966c2c4a44c779634e","analyzedAt":"2026-09-04T12:50:26.162Z","contentChangedAt":"2026-09-04T12:50:26.162Z","schemaVersion":2},"datasetVersion":"2026-09-11T21:17:09.523Z"}