{"record":{"id":"e17b397fbd8447d3","repo":"apache/cassandra","slug":"unknown-column-name-during-deserialization","errorCode":null,"errorMessage":"Unknown column <name> during deserialization","messagePattern":"Unknown column <name> during deserialization","errorType":"exception","errorClass":"java.lang.RuntimeException","httpStatus":null,"severity":"critical","filePath":"src/java/org/apache/cassandra/db/Columns.java","lineNumber":513,"sourceCode":"\n        public Columns deserialize(DataInputPlus in, TableMetadata metadata) throws IOException\n        {\n            int length = in.readUnsignedVInt32();\n            try (BTree.FastBuilder<ColumnMetadata> builder = BTree.fastBuilder())\n            {\n                for (int i = 0; i < length; i++)\n                {\n                    ByteBuffer name = ByteBufferUtil.readWithVIntLength(in);\n                    ColumnMetadata column = metadata.getColumn(name);\n                    if (column == null)\n                    {\n                        // If we don't find the definition, it could be we have data for a dropped column, and we shouldn't\n                        // fail deserialization because of that. So we grab a \"fake\" ColumnMetadata that ensure proper\n                        // deserialization. The column will be ignore later on anyway.\n                        column = metadata.getDroppedColumn(name);\n\n                        if (column == null)\n                            throw new RuntimeException(\"Unknown column \" + UTF8Type.instance.getString(name) + \" during deserialization\");\n                    }\n                    builder.add(column);\n                }\n                return new Columns(builder.build());\n            }\n        }\n\n        /**\n         * If both ends have a pre-shared superset of the columns we are serializing, we can send them much\n         * more efficiently. Both ends must provide the identically same set of columns.\n         */\n        public void serializeSubset(Collection<ColumnMetadata> columns, Columns superset, DataOutputPlus out) throws IOException\n        {\n            /**\n             * We weight this towards small sets, and sets where the majority of items are present, since\n             * we expect this to mostly be used for serializing result sets.\n             *\n             * For supersets with fewer than 64 columns, we encode a bitmap of *missing* columns,","sourceCodeStart":495,"sourceCodeEnd":531,"githubUrl":"https://github.com/apache/cassandra/blob/88fd0f6a0eaed8943f05ac9e8f947882b8ddc8f1/src/java/org/apache/cassandra/db/Columns.java#L495-L531","documentation":"Thrown by Columns.Serializer.deserialize when a serialized column name cannot be resolved against the table metadata. The serializer first tries the regular columns and then, as a lenient fallback, the dropped-column registry; if neither knows the name, the on-disk data references a column this node's schema cannot account for. This indicates schema metadata and SSTable data are out of sync.","triggerScenarios":"Reading (deserializing) a partition whose stored cell column identifier is not present in TableMetadata and has no DroppedColumn entry; typically during a read/compaction/streaming after an altered schema.","commonSituations":"Schema restored from a snapshot taken after the data was written; dropped column without proper flush/GC of old data on another node; schema disagreement during rolling upgrades; manually edited system_schema tables.","solutions":["Run nodetool describecluster / check schema versions and resolve schema disagreement across nodes.","Verify the column was dropped correctly (ALTER TABLE ... DROP) so it is registered in dropped_columns; re-add then drop the column if the history was lost.","Restore schema from the same backup generation as the data, or run a repair/scrub after aligning schemas.","If unrecoverable, remove or rewrite the affected SSTables (nodetool scrub / garbage collect) after backing them up."],"exampleFix":"// before: reading data written under an older schema that lost its dropped_columns entry\n// cqlsh> DESCRIBE TABLE t;  -- column 'foo' missing, no dropped_columns record\n// after\n// cqlsh> ALTER TABLE t ADD foo int;  -- recreate at the original type\n// cqlsh> ALTER TABLE t DROP foo;     -- register it in dropped_columns, then retry the read","handlingStrategy":"try-catch","validationCode":"// Compare schema versions before reads after schema changes\nMetadata metadata = session.getMetadata().getKeyspaces().get(keyspace);\nif (metadata == null || !metadata.getTables().containsKey(table))\n    throw new IllegalStateException(\"schema mismatch for \" + keyspace + \".\" + table);","typeGuard":null,"tryCatchPattern":"try { resultSet = session.execute(read); }\ncatch (RuntimeException e) {\n    if (e.getMessage() != null && e.getMessage().contains(\"Unknown column\")) {\n        // schema/data mismatch: force schema refresh or repair\n        cluster.refreshSchema();\n    } else throw e;\n}","preventionTips":["Always ALTER TABLE ... DROP columns properly (never delete schema rows manually) so dropped_columns history is preserved.","Check nodetool describecluster schema agreement after schema changes before resuming traffic.","Flush and repair after schema migrations on all nodes.","Restore schema and data backups from the same generation."],"tags":["cassandra","deserialization","schema","sstable"],"backgroundTag":"schema-validation-failed","analyzedSha":"88fd0f6a0eaed8943f05ac9e8f947882b8ddc8f1","analyzedAt":"2026-09-10T07:29:22.284Z","contentChangedAt":"2026-09-10T07:29:22.284Z","schemaVersion":2},"datasetVersion":"2026-09-14T16:17:12.679Z"}