{"record":{"id":"588893149a510a57","repo":"apache/beam","slug":"schema-fields-count-s-does-not-fit-columnsmapping-count-s","errorCode":null,"errorMessage":"Schema fields count: '%s' does not fit columnsMapping count: '%s'","messagePattern":"Schema fields count: '(.+?)' does not fit columnsMapping count: '(.+?)'","errorType":"exception","errorClass":"IllegalStateException","httpStatus":null,"severity":"error","filePath":"sdks/java/extensions/sql/src/main/java/org/apache/beam/sdk/extensions/sql/meta/provider/bigtable/BigtableTable.java","lineNumber":190,"sourceCode":"              + \" 'googleapis.com/bigtable/projects/projectId/instances/instanceId/tables/tableId'\"\n              + \" but was: \"\n              + location);\n    }\n  }\n\n  private static void validateColumnsMapping(\n      Map<String, Set<String>> columnsMapping, Schema schema) {\n    validateColumnsMappingCount(columnsMapping, schema);\n    validateColumnsMappingFields(columnsMapping, schema);\n  }\n\n  private static void validateColumnsMappingCount(\n      Map<String, Set<String>> columnsMapping, Schema schema) {\n    int mappingCount = columnsMapping.values().stream().mapToInt(Set::size).sum();\n    // Don't count the key field\n    int qualifiersCount = schema.getFieldCount() - 1;\n    if (qualifiersCount != mappingCount) {\n      throw new IllegalStateException(\n          String.format(\n              \"Schema fields count: '%s' does not fit columnsMapping count: '%s'\",\n              qualifiersCount, mappingCount));\n    }\n  }\n\n  private static void validateColumnsMappingFields(\n      Map<String, Set<String>> columnsMapping, Schema schema) {\n    Set<String> allMappingQualifiers =\n        columnsMapping.values().stream().flatMap(Collection::stream).collect(toSet());\n\n    Set<String> schemaFieldNames =\n        schema.getFieldNames().stream().filter(field -> !KEY.equals(field)).collect(toSet());\n\n    if (!schemaFieldNames.equals(allMappingQualifiers)) {\n      throw new IllegalStateException(\n          String.format(\n              \"columnsMapping '%s' does not fit to schema field names '%s'\",","sourceCodeStart":172,"sourceCodeEnd":208,"githubUrl":"https://github.com/apache/beam/blob/12126d8942aaf848030c478b4c6a28c6af861c66/sdks/java/extensions/sql/src/main/java/org/apache/beam/sdk/extensions/sql/meta/provider/bigtable/BigtableTable.java#L172-L208","documentation":"In flat schema mode the TBLPROPERTIES columnsMapping must map exactly one Bigtable qualifier per non-key schema field. validateColumnsMappingCount sums all mapped qualifiers and compares with schema field count minus the key; a mismatch throws IllegalStateException.","triggerScenarios":"Total number of qualifiers across all column families in columnsMapping differs from (number of schema fields - 1): extra mapped qualifiers, missing ones, or schema fields added/removed without updating columnsMapping.","commonSituations":"Adding a new column to the schema but forgetting to add it to columnsMapping; listing the same qualifier twice across families inflating the count; including the 'key' field in columnsMapping.","solutions":["Make columnsMapping contain exactly one qualifier per non-key schema field, matching counts on both sides.","After changing the schema, update the columnsMapping TBLPROPERTIES in the same change.","Remove the 'key' field from columnsMapping (it is never counted as a qualifier)."],"exampleFix":"-- before (schema has key, val1, val2; mapping only has 1 qualifier)\nTBLPROPERTIES '{\"columnsMapping\": {\"cf\": [\"val1\"]}}'\n\n-- after\nTBLPROPERTIES '{\"columnsMapping\": {\"cf\": [\"val1\", \"val2\"]}}'","handlingStrategy":"validation","validationCode":"Schema schema = table.getSchema();\nMap<String, Set<String>> mapping = parseColumnsMapping(props);\nint qualifiers = mapping.values().stream().mapToInt(Set::size).sum();\nif (schema.getFieldCount() - 1 != qualifiers) {\n  throw new IllegalArgumentException(\"columnsMapping qualifier count must equal schema fields minus the key\");\n}","typeGuard":null,"tryCatchPattern":"try {\n  BigtableTable t = new BigtableTable(table);\n} catch (IllegalStateException e) {\n  if (e.getMessage().contains(\"does not fit columnsMapping count\")) {\n    // align schema fields and columnsMapping, then retry\n  }\n}","preventionTips":["Change schema and columnsMapping together in one commit/DDL update.","Never list the 'key' field inside columnsMapping.","Write a startup validation that compares field count minus key with mapping size."],"tags":["java","beam-sql","bigtable","schema","configuration"],"backgroundTag":"schema-validation-failed","analyzedSha":"12126d8942aaf848030c478b4c6a28c6af861c66","analyzedAt":"2026-09-13T01:50:10.254Z","contentChangedAt":"2026-09-13T01:50:10.254Z","schemaVersion":2},"datasetVersion":"2026-09-14T16:17:12.679Z"}