apache/beam · error · java.lang.RuntimeException

Error decoding payload

Error message

Error decoding payload

What it means

ExpansionService.decodeConfigObjectRow() decodes the ExternalConfigurationPayload's serialized bytes into a Row using a RowCoder built from the payload's schema; an IOException during decode is wrapped in this RuntimeException. It means the payload bytes do not match the payload schema.

Source

Thrown at sdks/java/expansion-service/src/main/java/org/apache/beam/sdk/expansion/service/ExpansionService.java:428

                (field) -> {
                  Preconditions.checkNotNull(field.getName());
                  if (field.getName().contains("_")) {
                    @Nullable String newName = camelCaseConverter.convert(field.getName());
                    assert newName != null
                        : "@AssumeAssertion(nullness): converter type is imprecise; it is"
                            + " nullness-preserving";
                    return field.withName(newName);
                  } else {
                    return field;
                  }
                })
            .collect(Schema.toSchema());

    Row configRow;
    try {
      configRow = RowCoder.of(payloadSchema).decode(payload.newInput());
    } catch (IOException e) {
      throw new RuntimeException("Error decoding payload", e);
    }
    return configRow;
  }

  /**
   * Attempt to create an instance of {@link ConfigT} from an {@link ExternalConfigurationPayload}.
   * If a schema is registered for {@link ConfigT} this method will attempt to ise it. Throws an
   * {@link IllegalArgumentException} if the schema in {@code payload} is not {@link
   * Schema#assignableTo(Schema) assignable to} the registered schema.
   *
   * <p>If no Schema is registered, {@link ConfigT} must have a zero-argument constructor and
   * setters corresponding to each field in the row encoded by {@code payload}. Note {@link ConfigT}
   * may have additional setters not represented in the {@code payload} schema.
   *
   * <p>Exposed for testing only. No backwards compatibility guarantees.
   */
  @VisibleForTesting
  public static <ConfigT> ConfigT payloadToConfig(

View on GitHub (pinned to 12126d8942)

Solutions

  1. Verify the payload bytes were encoded with the same schema embedded in the ExternalConfigurationPayload
  2. Regenerate the payload from the client SDK instead of hand-rolling protobuf bytes
  3. Align Beam SDK versions between sender and expansion service
  4. Log the payload schema and compare field-by-field with what the sender encodes

Example fix

// before: schema and bytes diverge after adding a field client-side
Row encoded with old schema but new schema declared in payload
// after: rebuild payload with schema-carrying helper on both sides
ExternalConfigurationPayload payload = ExternalConfigurationPayload.create(schema, bytes, null);
Defensive patterns

Strategy: validation

Validate before calling

// confirm payload schema matches the bytes before decode
Schema payloadSchema = SchemaTranslation.schemaFromProto(payload.getSchema());
Row roundTrip = RowCoder.of(payloadSchema).decode(new ByteArrayInputStream(RowCoder.of(payloadSchema).encode(row).array()));

Try / catch

try { row = decodeConfigObjectRow(payload, schema); } catch (RuntimeException e) { if (e.getMessage().equals("Error decoding payload")) { /* re-encode payload with the advertised schema */ } throw e; }

Prevention

When it happens

Trigger: Calling decodeConfigObjectRow (directly or via payloadRow/configRow) when RowCoder.of(payloadSchema).decode(payload.newInput()) throws IOException - typically byte/schema mismatch or truncated payload.

Common situations: Sender encodes config with a different schema version than the one advertised in the payload proto; corrupted payload in transit; hand-crafted FunctionSpec payloads in tests.

Understand the failure class

Background: Schema validation failed / invalid input schema: payload rejected because its shape doesn't match the expected schema — this error's family across 28 libraries.

Related errors

AI-assisted analysis of apache/beam@12126d8942 (2026-09-13). Data as JSON: /api/errors/0863bc10ae4900c7. Report an issue: GitHub.