apache/cassandra · error · IOException

Invalid Columns subset bytes; too many bits set: ${encoded}

Error message

Invalid Columns subset bytes; too many bits set: ${encoded}

What it means

UnfilteredDescriptor.readSmallColumnsSubset decodes a 'small' Columns subset from the sstable UnfilteredDescriptor block: a bitmask stored as an unsigned vint where each set bit refers to a column index in the row's column superset. If any bit beyond rowColumns.size() is set the encoding is impossible, so the reader throws IOException ('Invalid Columns subset bytes; too many bits set') indicating corrupt or miswritten sstable data (src/java/org/apache/cassandra/io/sstable/UnfilteredDescriptor.java:174), mirroring Columns.Serializer.deserializeSubset's corruption check.

Source

Thrown at src/java/org/apache/cassandra/io/sstable/UnfilteredDescriptor.java:174

        if (UnfilteredSerializer.hasAllColumns(flags))
            missingColumnsMask = 0;
        else if (rowColumns.size() < 64)
            readSmallColumnsSubset(dataReader);
        else
            readLargeColumnsSubset(dataReader);
    }

    /**
     * Columns.Serializer.deserializeSubset would build a Columns per row, so decode its wire format
     * here: an unsigned vint bitmask of the missing superset columns. rowColumns stays the superset,
     * and consumers filter with missingColumnsMask().
     */
    private void readSmallColumnsSubset(RandomAccessReader dataReader) throws IOException
    {
        long encoded = dataReader.readUnsignedVInt();
        // Mirrors the corruption check in Columns.Serializer.deserializeSubset.
        if ((encoded >>> rowColumns.size()) != 0)
            throw new IOException("Invalid Columns subset bytes; too many bits set: " + Long.toBinaryString(encoded));
        missingColumnsMask = encoded;
    }

    /**
     * Wire format per Columns.Serializer.serializeLargeSubset: an unsigned vint delta of
     * supersetCount - presentCount, then one unsigned vint superset index per column. The indices
     * name the present columns when presentCount is under half the superset, and the missing
     * columns otherwise. Decoding into reusable mask words leaves rowColumns as the superset, so
     * CellCursor never rebuilds its per-superset arrays.
     */
    private void readLargeColumnsSubset(RandomAccessReader dataReader) throws IOException
    {
        long encoded = dataReader.readUnsignedVInt();
        int supersetCount = rowColumns.size();
        if (encoded > supersetCount)
            throw new IOException("Invalid large Columns subset: missing count " + encoded + " of " + supersetCount);

        int delta = (int) encoded;

View on GitHub (pinned to 88fd0f6a0e)

Solutions

Run nodetool verify (or scrub) on the table to identify and quarantine the corrupt sstable
Restore the affected sstable(s) from a backup/snapshot or run an anticompaaction/repair to rebuild the data from other replicas
Check disk health (SMART, dmesg) for bit-rot and repair or replace the failing device
Ensure the reader and writer Cassandra versions use the same sstable format; upgrade the reader if the file uses a newer format

Example fix

// before
// silently trusting on-disk bytes
columns = Columns.Serializer.deserializeSubset(...);
// after
if ((encoded >>> supersetCount) != 0)
    throw new IOException("Invalid Columns subset bytes; too many bits set: " + Long.toBinaryString(encoded)); // corrupt sstable -> scrub/repair

Defensive patterns

Strategy: try-catch

Validate before calling

long encoded = dataReader.readUnsignedVInt();
if ((encoded >>> rowColumns.size()) != 0)
    throw new IOException("corrupt small columns subset: " + Long.toBinaryString(encoded));

Try / catch

try {
    readSmallColumnsSubset(reader);
} catch (IOException e) {
    // quarantine sstable and fall back to another replica / backup
}

Prevention

Schedule regular nodetool verify to catch corrupt descriptors early
Use snapshots for all sstable movement; avoid manual partial copies
Replace hardware showing repeated single-bit corruption patterns

When it happens

Trigger: Reading an sstable (loadCommonRowFields path) whose descriptor block contains a small-subset mask with high bits set beyond the column count — i.e. the bytes at that offset are not a valid subset mask: bit-flip disk corruption, a truncated/misaligned read at the wrong offset, or a file written by an incompatible writer.

Common situations: Disk corruption or failing hardware on an sstable; copying/moving sstable files without snapshots leading to partial data; restoring a backup at the wrong position/offset; version mismatch where a reader misinterprets a differently encoded block from another Cassandra version.

Related errors

AI-assisted analysis of apache/cassandra@88fd0f6a0e (2026-09-10). Data as JSON: /api/errors/31153ee76bdc6d7a. Report an issue: GitHub.