{"record":{"id":"3990b75dc23c6b19","repo":"apache/beam","slug":"unexpected-field-fieldname-in-top-level-schema-for-pubsub","errorCode":null,"errorMessage":"Unexpected field '${fieldName}' in top level schema for Pubsub message. Top level schema should only contain 'timestamp', 'attributes', and 'payload' fields","messagePattern":"Unexpected field '(.+?)' in top level schema for Pubsub message\\. Top level schema should only contain 'timestamp', 'attributes', and 'payload' fields","errorType":"exception","errorClass":"IllegalArgumentException","httpStatus":null,"severity":"error","filePath":"sdks/java/io/google-cloud-platform/src/main/java/org/apache/beam/sdk/io/gcp/pubsub/PubsubMessageToRow.java","lineNumber":236,"sourceCode":"          (k, v) -> rows.add(Row.withSchema(ATTRIBUTE_ARRAY_ENTRY_SCHEMA).attachValues(k, v)));\n      return rows.build();\n    }\n\n    /** Get the value for a field int the order they're specified in the nested schema. */\n    private @Nullable Object getValueForFieldNestedSchema(\n        Schema.Field field,\n        Instant timestamp,\n        @Nullable Map<String, String> attributeMap,\n        byte[] payload) {\n      switch (field.getName()) {\n        case TIMESTAMP_FIELD:\n          return timestamp;\n        case ATTRIBUTES_FIELD:\n          return handleAttributes(attributeMap);\n        case PAYLOAD_FIELD:\n          return maybeDeserialize(payload);\n        default:\n          throw new IllegalArgumentException(\n              \"Unexpected field '\"\n                  + field.getName()\n                  + \"' in top level schema\"\n                  + \" for Pubsub message. Top level schema should only contain \"\n                  + \"'timestamp', 'attributes', and 'payload' fields\");\n      }\n    }\n\n    @ProcessElement\n    public void processElement(\n        @Element PubsubMessage element, @Timestamp Instant timestamp, MultiOutputReceiver o) {\n      try {\n        List<@Nullable Object> values =\n            messageSchema.getFields().stream()\n                .<@Nullable Object>map(\n                    field ->\n                        getValueForFieldNestedSchema(\n                            field, timestamp, element.getAttributeMap(), element.getPayload()))","sourceCodeStart":218,"sourceCodeEnd":254,"githubUrl":"https://github.com/apache/beam/blob/12126d8942aaf848030c478b4c6a28c6af861c66/sdks/java/io/google-cloud-platform/src/main/java/org/apache/beam/sdk/io/gcp/pubsub/PubsubMessageToRow.java#L218-L254","documentation":"PubsubMessageToRow supports nested-schema mode where the top-level Row schema may only have the fields 'timestamp', 'attributes', and 'payload'. getValueForFieldNestedSchema hits an unknown field name and throws IllegalArgumentException, listing the only allowed top-level fields.","triggerScenarios":"Supplying a top-level schema (via PubsubMessageToRow / PubsubIO.readMessagesWithAttributes with a schema) containing any field other than timestamp/attributes/payload in nested (non-flat) mode.","commonSituations":"Users define a schema where the message body fields are placed at the top level instead of nested under 'payload', or they rename the standard fields, or they confuse flat vs nested schema modes.","solutions":["Restructure the top-level schema to contain only 'timestamp', 'attributes', and 'payload' fields; put custom fields inside 'payload'.","If you intended a flat mapping of attributes/payload to custom fields, use the flat schema mode (attribute/payload attribute configuration) instead of nested mode.","Fix field-name typos or casing (e.g. 'Timestamp' -> 'timestamp')."],"exampleFix":"// before\nSchema.of(Field.of(\"event_time\", ...), Field.of(\"attributes\", ...)) // unknown top-level field\n// after\nSchema.of(\n  Field.of(\"timestamp\", ...),\n  Field.of(\"attributes\", ...),\n  Field.of(\"payload\", Schema.of(Field.of(\"event_time\", ...))))","handlingStrategy":"validation","validationCode":"java.util.Set<String> allowed = new java.util.HashSet<>(java.util.Arrays.asList(\"timestamp\", \"attributes\", \"payload\"));\nfor (Schema.Field f : schema.getFields()) {\n  if (!allowed.contains(f.getName())) throw new IllegalArgumentException(\"Top-level field not allowed: \" + f.getName());\n}","typeGuard":"boolean isNestedTopLevelField = java.util.Arrays.asList(\"timestamp\",\"attributes\",\"payload\").contains(fieldName);","tryCatchPattern":"try {\n  Row value = values(message, schema);\n} catch (IllegalArgumentException e) {\n  // fix schema definition; log field name from message\n}","preventionTips":["Only ever place timestamp/attributes/payload at the top level in nested mode","Put custom fields inside the payload sub-schema","Double-check field names and casing against the spec"],"tags":["java","pubsub","beam","schema","illegal-argument"],"backgroundTag":"schema-validation-failed","analyzedSha":"12126d8942aaf848030c478b4c6a28c6af861c66","analyzedAt":"2026-09-13T01:50:10.254Z","contentChangedAt":"2026-09-13T01:50:10.254Z","schemaVersion":2},"datasetVersion":"2026-09-14T16:17:12.679Z"}