What causes "newDictionary must have the same position count" in prestodb/presto?

Ensure the replacement dictionary has the same position count as the original, remapping ids if entries were removed.

prestodb/presto · error · IllegalArgumentException

newDictionary must have the same position count

Error message

newDictionary must have the same position count

What it means

DictionaryBlock.createProjection replaces the block's dictionary with a new one while keeping the same id array. This only works if the new dictionary has exactly the same number of positions as the old one, since the existing ids index directly into it. The library throws IllegalArgumentException when the position counts differ, because otherwise ids could point past the end of the new dictionary.

Source

Thrown at presto-common/src/main/java/com/facebook/presto/common/block/DictionaryBlock.java:514

    {
        return format("DictionaryBlock(%d){positionCount=%d,dictionary=%s}", hashCode(), getPositionCount(), dictionary.toString());
    }

    @Override
    public Block getLoadedBlock()
    {
        Block loadedDictionary = dictionary.getLoadedBlock();

        if (loadedDictionary == dictionary) {
            return this;
        }
        return new DictionaryBlock(idsOffset, getPositionCount(), loadedDictionary, ids, false, randomDictionaryId());
    }

    public Block createProjection(Block newDictionary)
    {
        if (newDictionary.getPositionCount() != dictionary.getPositionCount()) {
            throw new IllegalArgumentException("newDictionary must have the same position count");
        }

        // if the new dictionary is lazy be careful to not materialize it
        if (newDictionary instanceof LazyBlock) {
            return new LazyBlock(positionCount, (block) -> {
                Block newDictionaryBlock = newDictionary.getBlock(0);
                Block newBlock = createProjection(newDictionaryBlock);
                block.setBlock(newBlock);
            });
        }
        if (newDictionary instanceof RunLengthEncodedBlock) {
            RunLengthEncodedBlock rle = (RunLengthEncodedBlock) newDictionary;
            return new RunLengthEncodedBlock(rle.getValue(), positionCount);
        }

        // unwrap dictionary in dictionary
        int[] newIds = new int[positionCount];
        for (int position = 0; position < positionCount; position++) {

View on GitHub (pinned to 55bb57d202)

Solutions

Ensure the replacement dictionary has the same position count as the original, remapping ids if entries were removed.
If the dictionaries differ, first call createProjection with an explicit id mapping or rebuild the DictionaryBlock with newIds computed from the new dictionary.
Wrap the call in a check: if (newDictionary.getPositionCount() != block.getDictionary().getPositionCount()) build a new block instead.

Example fix

// before
Block projected = dictionaryBlock.createProjection(filteredDictionary);
// after
if (filteredDictionary.getPositionCount() != dictionaryBlock.getDictionary().getPositionCount()) {
    throw new IllegalStateException("projection requires same-size dictionary; remap ids instead");
}
Block projected = dictionaryBlock.createProjection(filteredDictionary);

Defensive patterns

Strategy: validation

Validate before calling

if (newDictionary.getPositionCount() != dictionaryBlock.getDictionary().getPositionCount()) {
    throw new IllegalArgumentException("cannot project: dictionary sizes differ (" +
        newDictionary.getPositionCount() + " vs " + dictionaryBlock.getDictionary().getPositionCount() + ")");
}

Type guard

boolean isProjectable(DictionaryBlock block, Block newDictionary) {
    return newDictionary.getPositionCount() == block.getDictionary().getPositionCount();
}

Try / catch

try {
    projected = dictionaryBlock.createProjection(newDictionary);
} catch (IllegalArgumentException e) {
    if (e.getMessage().contains("same position count")) {
        projected = rebuildDictionaryBlock(dictionaryBlock, newDictionary); // remap ids
    } else { throw e; }
}

Prevention

Always derive the new dictionary from the same position set as the original.
Assert dictionary position-count equality in unit tests for dictionary-transforming operators.
Prefer remapping ids over reusing them when compaction changes the dictionary.

When it happens

Trigger: Calling DictionaryBlock.createProjection(newDictionary) where newDictionary.getPositionCount() != dictionary.getPositionCount(); e.g. projecting a dictionary block onto a dictionary with fewer or more entries.

Common situations: Re-encoding a dictionary after filtering or compacting positions without remapping ids; combining dictionary-encoded blocks from different sources whose dictionaries were built independently; operators that rebuild dictionaries with deduplication changing the entry count.

Related errors

AI-assisted analysis of prestodb/presto@55bb57d202 (2026-09-04). Data as JSON: /api/errors/63f44a7f81053404. Report an issue: GitHub.