{"record":{"id":"573adb17045adf18","repo":"prestodb/presto","slug":"reference-to-a-non-existent-key","errorCode":null,"errorMessage":"reference to a non-existent key","messagePattern":"reference to a non-existent key","errorType":"exception","errorClass":"java.lang.IllegalStateException","httpStatus":null,"severity":"error","filePath":"presto-common/src/main/java/com/facebook/presto/common/Page.java","lineNumber":312,"sourceCode":"            try {\n                Block compactDictionary = dictionaryBlock.getDictionary().copyPositions(dictionaryPositionsToCopy, 0, numberOfIndexes);\n                outputDictionaryBlocks.add(new DictionaryBlock(positionCount, compactDictionary, newIds, true, newDictionaryId));\n            }\n            catch (UnsupportedOperationException e) {\n                // ignore if copy positions is not supported for the dictionary\n                outputDictionaryBlocks.add(dictionaryBlock);\n            }\n        }\n        return outputDictionaryBlocks;\n    }\n\n    private static int[] getNewIds(int positionCount, DictionaryBlock dictionaryBlock, int[] remapIndex)\n    {\n        int[] newIds = new int[positionCount];\n        for (int i = 0; i < positionCount; i++) {\n            int newId = remapIndex[dictionaryBlock.getId(i)];\n            if (newId == -1) {\n                throw new IllegalStateException(\"reference to a non-existent key\");\n            }\n            newIds[i] = newId;\n        }\n        return newIds;\n    }\n\n    /**\n     * Returns a page that assures all data is in memory.\n     * May return the same page if all page data is already in memory.\n     * <p>\n     * This allows streaming data sources to skip sections that are not\n     * accessed in a query.\n     */\n    public Page getLoadedPage()\n    {\n        for (int i = 0; i < blocks.length; i++) {\n            Block loaded = blocks[i].getLoadedBlock();\n            if (loaded != blocks[i]) {","sourceCodeStart":294,"sourceCodeEnd":330,"githubUrl":"https://github.com/prestodb/presto/blob/55bb57d202de3b926896fa966c2c4a44c779634e/presto-common/src/main/java/com/facebook/presto/common/Page.java#L294-L330","documentation":"getNewIds remaps each dictionary id in a DictionaryBlock through remapIndex built from the positions of the dictionary that will be retained. A remap value of -1 means the block references a dictionary entry that was not marked for retention, so the compacted dictionary would be missing an entry. The library throws IllegalStateException instead of producing a corrupt page.","triggerScenarios":"Page.compactBlocks where a DictionaryBlock's ids point at dictionary positions that dictionaryPositionsToCopy did not include — typically when blocks claim the same DictionarySourceId but actually have divergent dictionary contents, or remapIndex was computed against a different dictionary than the block's.","commonSituations":"Bugs in custom Block implementations that lie about their DictionarySourceId; dictionary id collisions from randomDictionaryId after deserializing pages across versions.","solutions":["Verify all blocks with equal DictionarySourceId truly share identical dictionary contents; fix the producer that fabricates/collides dictionary ids","Decode the DictionaryBlock into a plain block (getLoadedBlock/copyPositions) before compaction","Rebuild the Page from source data so dictionary and ids are regenerated consistently","Check for Presto version/serialization issues that duplicate dictionary ids and upgrade"],"exampleFix":"// before\n// block claims same source id but has extra ids -> remapIndex[id] == -1\npage.compactBlocks();\n// after\nBlock decoded = dictionaryBlock.getLoadedBlock(); // or copy to plain block\nPage fixed = new Page(decoded, otherBlocks);\nfixed.compactBlocks();","handlingStrategy":"validation","validationCode":"int[] remap = buildRemapIndex(dictionaryPositionsToCopy);\nfor (int i = 0; i < dictionaryBlock.getPositionCount(); i++) {\n    if (remap[dictionaryBlock.getId(i)] == -1) {\n        throw new IllegalStateException(\"dictionary id \" + dictionaryBlock.getId(i) + \" not retained; decode block first\");\n    }\n}","typeGuard":"boolean idsWithinRetained(DictionaryBlock block, boolean[] retained) {\n    for (int i = 0; i < block.getPositionCount(); i++) {\n        if (!retained[block.getId(i)]) return false;\n    }\n    return true;\n}","tryCatchPattern":"try {\n    return compactDictionaryPage(page);\n} catch (IllegalStateException e) {\n    if (!e.getMessage().contains(\"non-existent key\")) throw e;\n    return decodeToPlainBlocks(page); // fallback path\n}","preventionTips":["Never fake DictionarySourceId equality in custom blocks","Verify dictionary contents match whenever ids are equal","Decode to plain blocks when dictionary provenance is uncertain"],"tags":["presto","dictionary-encoding","block-compaction","corrupt-state"],"backgroundTag":"dictionary-source-id-mismatch","analyzedSha":"55bb57d202de3b926896fa966c2c4a44c779634e","analyzedAt":"2026-09-04T12:50:26.162Z","contentChangedAt":"2026-09-04T12:50:26.162Z","schemaVersion":2},"datasetVersion":"2026-09-11T21:17:09.523Z"}