{"record":{"id":"d84ea2f001e7fa7f","repo":"apache/iceberg","slug":"id-column-not-found-colname","errorCode":null,"errorMessage":"ID column not found: ${colName}","messagePattern":"ID column not found: (.+?)","errorType":"exception","errorClass":"IllegalArgumentException","httpStatus":null,"severity":"error","filePath":"kafka-connect/kafka-connect/src/main/java/org/apache/iceberg/connect/data/RecordUtils.java","lineNumber":125,"sourceCode":"\n    long targetFileSize =\n        PropertyUtil.propertyAsLong(\n            tableProps,\n            TableProperties.WRITE_TARGET_FILE_SIZE_BYTES,\n            TableProperties.WRITE_TARGET_FILE_SIZE_BYTES_DEFAULT);\n\n    Set<Integer> identifierFieldIds = table.schema().identifierFieldIds();\n\n    // override the identifier fields if the config is set\n    List<String> idCols = config.tableConfig(tableReference.identifier().toString()).idColumns();\n    if (!idCols.isEmpty()) {\n      identifierFieldIds =\n          idCols.stream()\n              .map(\n                  colName -> {\n                    NestedField field = table.schema().findField(colName);\n                    if (field == null) {\n                      throw new IllegalArgumentException(\"ID column not found: \" + colName);\n                    }\n                    return field.fieldId();\n                  })\n              .collect(Collectors.toSet());\n    }\n\n    FileWriterFactory<Record> writerFactory;\n    if (identifierFieldIds == null || identifierFieldIds.isEmpty()) {\n      writerFactory =\n          new GenericFileWriterFactory.Builder(table)\n              .dataSchema(table.schema())\n              .dataFileFormat(format)\n              .writerProperties(tableProps)\n              .build();\n    } else {\n      writerFactory =\n          new GenericFileWriterFactory.Builder(table)\n              .dataSchema(table.schema())","sourceCodeStart":107,"sourceCodeEnd":143,"githubUrl":"https://github.com/apache/iceberg/blob/86d9c8fc543e7c56c9f624eb725f76c9baff9570/kafka-connect/kafka-connect/src/main/java/org/apache/iceberg/connect/data/RecordUtils.java#L107-L143","documentation":"When building the table writer with identifier (ID) columns, each configured ID column is looked up in the target Iceberg table schema. If schema.findField returns null the column does not exist and an IllegalArgumentException is thrown.","triggerScenarios":"Creating/updating a table from the Kafka Connect sink with ID columns (iceberg.columns config / identifier fields, usually for upserts) where a configured column name is misspelled or absent from the schema.","commonSituations":"Case-sensitivity mismatches (schema uses 'UserID', config says 'userid'); column added later upstream but table auto-evolution off; typos in iceberg.columns for key fields.","solutions":["Fix the configured column name to exactly match a field in the Iceberg table schema (names are case-sensitive).","Verify with SHOW CREATE TABLE / Spark DESCRIBE which columns exist; enable auto-create/evolution (iceberg.tables.auto-evolution.enabled=true) if upstream added new columns.","Ensure the key-field config aligns with fields produced by the converter."],"exampleFix":"// before\n\"iceberg.columns\": \"user_id,ordertime\"\n// after (schema field is actually 'userId')\n\"iceberg.columns\": \"userId,ordertime\"","handlingStrategy":"validation","validationCode":"for (String col : idCols) {\n  if (table.schema().findField(col) == null)\n    throw new ConfigException(\"ID column missing from table schema: \" + col);\n}","typeGuard":null,"tryCatchPattern":"try { writer = createTableWriter(...); }\ncatch (IllegalArgumentException e) { LOG.error(\"config/schema mismatch: {}\", e.getMessage()); throw e; }","preventionTips":["Compare configured ID columns against DESCRIBE output before enabling upserts","Remember field names are case-sensitive","Enable auto-evolution if the upstream schema changes"],"tags":["kafka-connect","iceberg-schema","identifier-column"],"backgroundTag":"resource-not-found","analyzedSha":"86d9c8fc543e7c56c9f624eb725f76c9baff9570","analyzedAt":"2026-09-12T00:46:39.097Z","contentChangedAt":"2026-09-12T00:46:39.097Z","schemaVersion":2},"datasetVersion":"2026-09-23T08:17:48.524Z"}