{"record":{"id":"b2fb77e8318b6263","repo":"apache/beam","slug":"columnsmapping-s-does-not-fit-to-schema-field-names-s","errorCode":null,"errorMessage":"columnsMapping '%s' does not fit to schema field names '%s'","messagePattern":"columnsMapping '(.+?)' does not fit to schema field names '(.+?)'","errorType":"exception","errorClass":"IllegalStateException","httpStatus":null,"severity":"error","filePath":"sdks/java/extensions/sql/src/main/java/org/apache/beam/sdk/extensions/sql/meta/provider/bigtable/BigtableTable.java","lineNumber":206,"sourceCode":"    int qualifiersCount = schema.getFieldCount() - 1;\n    if (qualifiersCount != mappingCount) {\n      throw new IllegalStateException(\n          String.format(\n              \"Schema fields count: '%s' does not fit columnsMapping count: '%s'\",\n              qualifiersCount, mappingCount));\n    }\n  }\n\n  private static void validateColumnsMappingFields(\n      Map<String, Set<String>> columnsMapping, Schema schema) {\n    Set<String> allMappingQualifiers =\n        columnsMapping.values().stream().flatMap(Collection::stream).collect(toSet());\n\n    Set<String> schemaFieldNames =\n        schema.getFieldNames().stream().filter(field -> !KEY.equals(field)).collect(toSet());\n\n    if (!schemaFieldNames.equals(allMappingQualifiers)) {\n      throw new IllegalStateException(\n          String.format(\n              \"columnsMapping '%s' does not fit to schema field names '%s'\",\n              allMappingQualifiers, schemaFieldNames));\n    }\n  }\n\n  private BigtableIO.Read readTransform() {\n    BigtableIO.Read readTransform =\n        BigtableIO.read().withProjectId(projectId).withInstanceId(instanceId).withTableId(tableId);\n    if (!emulatorHost.isEmpty()) {\n      readTransform = readTransform.withEmulator(emulatorHost);\n    }\n    return readTransform;\n  }\n\n  private PTransform<PCollection<com.google.bigtable.v2.Row>, PCollection<Row>> bigtableRowToRow() {\n    return useFlatSchema\n        ? new BigtableRowToBeamRowFlat(schema, columnsMapping)","sourceCodeStart":188,"sourceCodeEnd":224,"githubUrl":"https://github.com/apache/beam/blob/12126d8942aaf848030c478b4c6a28c6af861c66/sdks/java/extensions/sql/src/main/java/org/apache/beam/sdk/extensions/sql/meta/provider/bigtable/BigtableTable.java#L188-L224","documentation":"The set of qualifiers in columnsMapping must equal the set of non-key schema field names exactly. validateColumnsMappingFields throws IllegalStateException when the two sets differ — even if counts match, names must correspond one-to-one.","triggerScenarios":"A mapped qualifier name does not match any schema field name (or vice versa): typos in either the schema or columnsMapping, a field renamed in one place only, or mapping to Bigtable qualifier names that differ from schema field names.","commonSituations":"Schema field 'user_name' mapped as qualifier 'userName'; adding a field to schema and reusing an old mapping; expecting qualifier names in Bigtable to differ from SQL column names (flat schema requires them to be identical sets).","solutions":["Rename mapped qualifiers (or schema fields) so the two sets are identical, ignoring the 'key' field.","Compare both lists (schema fields minus key vs all columnsMapping qualifiers) and fix typos.","If the Bigtable qualifier must differ from the SQL column name, flat-schema mode does not allow it — rename the column in Bigtable or use a non-flat access path via BigtableIO instead."],"exampleFix":"-- before\nCREATE EXTERNAL TABLE bt (key STRING, userName STRING)\nTBLPROPERTIES '{\"columnsMapping\": {\"cf\": [\"user_name\"]}}'\n\n-- after\nCREATE EXTERNAL TABLE bt (key STRING, user_name STRING)\nTBLPROPERTIES '{\"columnsMapping\": {\"cf\": [\"user_name\"]}}'","handlingStrategy":"validation","validationCode":"Set<String> mapped = mapping.values().stream().flatMap(Set::stream).collect(java.util.stream.Collectors.toSet());\nSet<String> fields = new HashSet<>(java.util.Arrays.asList(schema.getFieldNames()));\nfields.remove(\"key\");\nif (!fields.equals(mapped)) {\n  throw new IllegalArgumentException(\"columnsMapping qualifiers must exactly match schema fields (excluding key)\");\n}","typeGuard":null,"tryCatchPattern":"try {\n  BigtableTable t = new BigtableTable(table);\n} catch (IllegalStateException e) {\n  if (e.getMessage().contains(\"does not fit to schema field names\")) {\n    // rename qualifiers/fields until the sets match\n  }\n}","preventionTips":["Use identical names for schema fields and Bigtable qualifiers in flat-schema mode.","Watch out for snake_case vs camelCase when copying names between Bigtable and SQL DDL.","Re-run mapping validation after any column rename."],"tags":["java","beam-sql","bigtable","schema","configuration"],"backgroundTag":"schema-validation-failed","analyzedSha":"12126d8942aaf848030c478b4c6a28c6af861c66","analyzedAt":"2026-09-13T01:50:10.254Z","contentChangedAt":"2026-09-13T01:50:10.254Z","schemaVersion":2},"datasetVersion":"2026-09-14T16:17:12.679Z"}