{"record":{"id":"b1f460a97b942841","repo":"apache/beam","slug":"ignoring-unmatched-input-fields","errorCode":null,"errorMessage":"Ignoring unmatched input fields: {}","messagePattern":"Ignoring unmatched input fields: (.+?)","errorType":"console","errorClass":null,"httpStatus":null,"severity":"warning","filePath":"sdks/java/io/amazon-web-services2/src/main/java/org/apache/beam/sdk/io/aws2/sqs/SqsIO.java","lineNumber":537,"sourceCode":"            // make sure field types match\n            if (!sourceField.typesEqual(targetSchema.getField(targetIdx))) {\n              invalid.add(sourceField.getName());\n            }\n            fieldMapping[targetIdx] = i;\n          } else {\n            ignored.add(sourceField.getName());\n          }\n        }\n        checkState(\n            ignored.size() < sourceSchema.getFieldCount(),\n            \"No fields matched, expected %s but got %s\",\n            schema.getFieldNames(),\n            ignored);\n\n        checkState(invalid.isEmpty(), \"Detected incompatible types for input fields: {}\", invalid);\n\n        if (!ignored.isEmpty()) {\n          LOG.warn(\"Ignoring unmatched input fields: {}\", ignored);\n        }\n      }\n\n      @Override\n      public SendMessageBatchRequestEntry apply(String entryId, T input) {\n        Row row = toRow.apply(input);\n        Object[] values = new Object[fieldMapping.length];\n        values[0] = entryId;\n        for (int i = 0; i < values.length; i++) {\n          if (fieldMapping[i] >= 0) {\n            values[i] = row.getValue(fieldMapping[i]);\n          }\n        }\n        return fromRow.apply(Row.withSchema(schema).attachValues(values));\n      }\n    }\n\n    /** Result of {@link #writeBatches}. */","sourceCodeStart":519,"sourceCodeEnd":555,"githubUrl":"https://github.com/apache/beam/blob/12126d8942aaf848030c478b4c6a28c6af861c66/sdks/java/io/amazon-web-services2/src/main/java/org/apache/beam/sdk/io/aws2/sqs/SqsIO.java#L519-L555","documentation":"SqsIO's SchemaEntryMapper maps input elements to SQS SendMessageBatchRequestEntry fields via a Beam schema. Input fields that don't match any target schema field are not errors; they are collected as 'ignored' and logged with this warning so users know those fields are silently dropped from each message.","triggerScenarios":"Using SqsIO.write() with .withEntryMapper/schema mapping on a PCollection whose schema contains fields not referenced by any SQS entry attribute/body mapping — e.g. a Row with extra columns after an upstream schema evolution.","commonSituations":"Upstream producers added new fields; reusing a mapper config across pipeline versions; accidental extra columns from a JOIN in SQL-generated PCollections.","solutions":["Add mappings for the extra fields if their values must reach SQS","Or narrow the input schema to only fields the mapper consumes (e.g. via select()/Row projection)","Ignore safely if the fields are intentionally unused — the message is informational","Check the companion 'Detected incompatible types' error if fields are matched but mistyped"],"exampleFix":"// before: input has fields 'payload,ts,extra' but only payload,ts mapped\n// after: project out the unmapped field\nPCollection<Row> mapped = input.getSchema().match();\nmapped = input.apply(Select.fieldNames(\"payload\", \"ts\"));","handlingStrategy":"validation","validationCode":"// ensure input schema fields are all consumed by the mapper\nSet<String> mappedFields = mapperConfig.consumedFieldNames();\nSet<String> inputFields = new HashSet<>(input.getSchema().getFieldNames());\nSet<String> unmatched = new HashSet<>(inputFields);\nunmatched.removeAll(mappedFields);\nif (!unmatched.isEmpty()) throw new IllegalStateException(\"Unmapped fields: \" + unmatched);","typeGuard":null,"tryCatchPattern":null,"preventionTips":["Project input via Select.fieldNames(...) to exactly the mapped fields","Re-run mapping config review after upstream schema changes","Treat this warning as data-loss signal for the listed fields","Add schema compatibility tests between producers and the sink"],"tags":["sqs","schema","mapping","unused-fields"],"backgroundTag":"schema-validation-failed","analyzedSha":"12126d8942aaf848030c478b4c6a28c6af861c66","analyzedAt":"2026-09-13T01:50:10.254Z","contentChangedAt":"2026-09-13T01:50:10.254Z","schemaVersion":2},"datasetVersion":"2026-09-14T21:17:11.552Z"}