prestodb/presto · critical · IllegalArgumentException

Deserialized SingleMapBlock violates invariants: expected ha

Error message

Deserialized SingleMapBlock violates invariants: expected hashtable size %d, actual hashtable size %d

What it means

SingleMapBlockEncoding.readBlock validates that the deserialized hash table's length equals keyBlock.getPositionCount() * HASH_MULTIPLIER. The hash table is a fixed-size open-addressing structure sized proportionally to the entry count; any other size means the stream is malformed or from an incompatible producer, so readBlock throws IllegalArgumentException with expected vs actual size.

Source

Thrown at presto-common/src/main/java/com/facebook/presto/common/block/SingleMapBlockEncoding.java:79

    public Block readBlock(BlockEncodingSerde blockEncodingSerde, SliceInput sliceInput)
    {
        Block keyBlock = blockEncodingSerde.readBlock(sliceInput);
        Block valueBlock = blockEncodingSerde.readBlock(sliceInput);

        int hashTableLength = sliceInput.readInt();
        int[] hashTable = null;
        if (hashTableLength >= 0) {
            hashTable = new int[hashTableLength];
            sliceInput.readBytes(wrappedIntArray(hashTable));
        }

        if (keyBlock.getPositionCount() != valueBlock.getPositionCount()) {
            throw new IllegalArgumentException(
                    format("Deserialized SingleMapBlock violates invariants: key %d, value %d", keyBlock.getPositionCount(), valueBlock.getPositionCount()));
        }

        if (hashTable != null && keyBlock.getPositionCount() * HASH_MULTIPLIER != hashTable.length) {
            throw new IllegalArgumentException(
                    format("Deserialized SingleMapBlock violates invariants: expected hashtable size %d, actual hashtable size %d", keyBlock.getPositionCount() * HASH_MULTIPLIER, hashTable.length));
        }

        MapBlock mapBlock = MapBlock.createMapBlockInternal(
                0,
                1,
                Optional.empty(),
                new int[] {0, keyBlock.getPositionCount()},
                keyBlock,
                valueBlock,
                new HashTables(Optional.ofNullable(hashTable), 1));

        return new SingleMapBlock(0, 0, keyBlock.getPositionCount() * 2, mapBlock);
    }
}

View on GitHub (pinned to 55bb57d202)

Solutions

Align Presto versions across all cluster nodes to fix HASH_MULTIPLIER/encoding skew.
Regenerate corrupted spill or exchange data; check disk and network reliability.
If writing the encoding manually, size the hash table as positionCount * HASH_MULTIPLIER.
Retry the operation and enable checksums to surface corruption earlier.

Example fix

// before
int[] hashTable = new int[keyBlock.getPositionCount()]; // wrong size
// after
int[] hashTable = new int[keyBlock.getPositionCount() * HASH_MULTIPLIER];

Defensive patterns

Strategy: try-catch

Validate before calling

// writer-side pre-check
int expected = keyBlock.getPositionCount() * HASH_MULTIPLIER;
if (hashTables != null && hashTables.length != expected) {
    throw new IllegalStateException("hash table size " + hashTables.length + " != expected " + expected);
}

Try / catch

try {
    Block block = blockEncoding.readBlock(sliceInput);
} catch (IllegalArgumentException e) {
    if (e.getMessage().contains("expected hashtable size")) {
        throw new CorruptedPageException("Map block hashtable size mismatch — version skew or corrupted data", e);
    }
    throw e;
}

Prevention

Size hash tables as positionCount * HASH_MULTIPLIER whenever constructing MapBlocks.
Align Presto versions across nodes before rolling upgrades of spill/exchange formats.
Validate serialized payloads with checksums and regenerate corrupted data.

When it happens

Trigger: Deserializing a MapBlock where hashTables length != keyBlock.getPositionCount() * HASH_MULTIPLIER — corrupted spill/exchange bytes, wrong HASH_MULTIPLIER version skew between writer and reader, or a custom serializer emitting a wrong-sized hash table.

Common situations: Cluster nodes on different Presto versions exchanging pages, truncated spill files, hand-rolled connectors writing MapBlock encoding manually.

Related errors

AI-assisted analysis of prestodb/presto@55bb57d202 (2026-09-04). Data as JSON: /api/errors/c8933fa13a8547bc. Report an issue: GitHub.