prestodb/presto · error · java.lang.IllegalArgumentException

dictionarySourceIds must be the same

Error message

dictionarySourceIds must be the same

What it means

During Page compaction of dictionary-encoded blocks, all DictionaryBlocks being compacted must share the same underlying dictionary (verified via DictionarySourceId). compactRelatedBlocks builds one compacted dictionary from the first block, so if any other block was built from a different dictionary, the compaction would silently corrupt data; the library refuses by throwing this IllegalArgumentException.

Source

Thrown at presto-common/src/main/java/com/facebook/presto/common/Page.java:291

            if (remapIndex[position] == -1) {
                dictionaryPositionsToCopy[numberOfIndexes] = position;
                remapIndex[position] = numberOfIndexes;
                numberOfIndexes++;
            }
        }

        // entire dictionary is referenced
        if (numberOfIndexes == dictionarySize) {
            return blocks;
        }

        // compact the dictionaries
        int[] newIds = getNewIds(positionCount, firstDictionaryBlock, remapIndex);
        List<DictionaryBlock> outputDictionaryBlocks = new ArrayList<>(blocks.size());
        DictionaryId newDictionaryId = randomDictionaryId();
        for (DictionaryBlock dictionaryBlock : blocks) {
            if (!firstDictionaryBlock.getDictionarySourceId().equals(dictionaryBlock.getDictionarySourceId())) {
                throw new IllegalArgumentException("dictionarySourceIds must be the same");
            }

            try {
                Block compactDictionary = dictionaryBlock.getDictionary().copyPositions(dictionaryPositionsToCopy, 0, numberOfIndexes);
                outputDictionaryBlocks.add(new DictionaryBlock(positionCount, compactDictionary, newIds, true, newDictionaryId));
            }
            catch (UnsupportedOperationException e) {
                // ignore if copy positions is not supported for the dictionary
                outputDictionaryBlocks.add(dictionaryBlock);
            }
        }
        return outputDictionaryBlocks;
    }

    private static int[] getNewIds(int positionCount, DictionaryBlock dictionaryBlock, int[] remapIndex)
    {
        int[] newIds = new int[positionCount];
        for (int i = 0; i < positionCount; i++) {

View on GitHub (pinned to 55bb57d202)

Solutions

1. Ensure all DictionaryBlocks in the Page share one dictionary, or convert some blocks to non-dictionary blocks before compaction
2. Unwrap/decode DictionaryBlocks (e.g. via block.getLoadedBlock() or copying to lazy/plain blocks) before calling compactBlocks
3. Check DictionaryBlock.getDictionarySourceId() equality yourself before compacting and rebuild the Page consistently
4. Upgrade/regenerate the dictionary so blocks produced in the same compaction batch originate from one dictionary source

Example fix

// before
DictionaryBlock a = new DictionaryBlock(dictionaryA, idsA);
DictionaryBlock b = new DictionaryBlock(dictionaryB, idsB);
page.compactBlocks(); // throws
// after
Block aPlain = a.copyPositions(a.getRange(0, a.getPositionCount()), 0, a.getPositionCount()); // or decode
Page fixed = new Page(aPlain, b.copyPositions(...));
fixed.compactBlocks();

DictionaryId first = ((DictionaryBlock) blocks.get(0)).getDictionarySourceId();
for (Block b : blocks) {
    if (b instanceof DictionaryBlock && !first.equals(((DictionaryBlock) b).getDictionarySourceId())) {
        throw new IllegalArgumentException("mixed dictionaries; decode blocks before compaction");
    }
}

boolean sameDictionary(Block a, Block b) {
    return !(a instanceof DictionaryBlock) || !(b instanceof DictionaryBlock)
        || a.getDictionarySourceId().equals(b.getDictionarySourceId());
}

try {
    page = page.compactBlocks();
} catch (IllegalArgumentException e) {
    if (!e.getMessage().contains("dictionarySourceIds")) throw e;
    page = decodeToPlainBlocks(page); // fallback: materialize dictionaries
}

When it happens

Trigger: Calling Page.compactBlocks/compactRelatedBlocks on a Page whose Block array contains multiple DictionaryBlocks created from different dictionaries (different DictionaryId), e.g. after mixing dictionary-encoded segments from different sources into one Page.

Common situations: Operators/scan pipelines that concatenate pages from different splits where each split dictionary-encoded its column independently; column readers that re-dictionary-encode without merging dictionary ids.

Related errors

• reference to a non-existent key
• newDictionary must have the same position count
• blocks is empty
• Column does not have same position count (%s) as page (%s)
• QualifiedObjectName should have exactly 3 parts, found %s: %

AI-assisted analysis of prestodb/presto@55bb57d202 (2026-09-04). Data as JSON: /api/errors/bdc0348bba8a4849. Report an issue: GitHub.