apache/beam · error · java.lang.IllegalStateException

Encountered a Managed Transform that has an empty…

Error message

Encountered a Managed Transform that has an empty "transform_identifier": %n%s

What it means

Beam's Managed transform wraps an underlying transform (e.g. Iceberg, KafkaIO) whose identity is stored in a schema'd config Row under the 'transform_identifier' field. During pipeline translation the decoded config Row must contain a non-null identifier so the MANAGED_UNDERLYING_TRANSFORM_URN_KEY annotation can be attached; if it is null the pipeline definition is corrupt and translation aborts with this IllegalStateException.

Solutions

Regenerate the pipeline with the same (or newer) Beam version on both the construction and expansion side so the Managed payload includes 'transform_identifier'
Inspect the pipeline representation (e.g. --pipelineDump / proto) and verify the Managed transform's configuration Row has a non-null 'transform_identifier'
Replace the Managed transform with the direct underlying transform (e.g. KafkaIO, IcebergIO) to bypass Managed translation

Example fix

// before
Row config = Row.withSchema(configSchema).addValues(null).build(); // no transform_identifier
// after
Row config = Row.withSchema(configSchema).addValues("beam:transform:kafka:write_v1").addValues(otherCfg).build();

Defensive patterns

Strategy: validation

Validate before calling

Row cfg = RowCoder.of(configSchema).decode(payload.getConfigurationRow().newInput());
if (cfg.getString("transform_identifier") == null) throw new IllegalArgumentException("Managed payload missing transform_identifier");

Type guard

boolean hasIdentifier(Row row) { return row != null && row.getSchema().getFieldNames().contains("transform_identifier") && row.getString("transform_identifier") != null; }

Prevention

Keep Beam versions identical across graph construction, expansion, and runtime
Never hand-edit pipeline protos/JSON for Managed transforms
Prefer explicit underlying transforms when Managed support is uncertain

When it happens

Trigger: Translating a pipeline whose RunnerApi payload contains a Managed transform (urn beam:transform:managed:v1) whose configuration Row lacks the 'transform_identifier' field or holds null for it — typically from a hand-built or cross-SDK-generated proto payload, or an older SDK writing a Managed payload without the identifier.

Common situations: Running a pipeline graph produced by a different Beam version or SDK that serializes Managed transforms differently; manually editing or post-processing a pipeline JSON/proto and dropping the config field; a runner deserializing a graph from a stale saved artifact.

Understand the failure class

Background: "is required", "must be set", "missing required field": configuration validation errors across open-source libraries — this error's family across 36 libraries.

Related errors

AI-assisted analysis of apache/beam@12126d8942 (2026-09-13). Data as JSON: /api/errors/bc0bdd4799b09e83. Report an issue: GitHub.

Appendix: source

Thrown at sdks/java/core/src/main/java/org/apache/beam/sdk/util/construction/PTransformTranslation.java:546

                components.getEnvironmentIdFor(appliedPTransform.getResourceHints()));
          }
        }

        if (spec.getUrn().equals(BeamUrns.getUrn(SCHEMA_TRANSFORM))) {
          ExternalTransforms.SchemaTransformPayload payload =
              ExternalTransforms.SchemaTransformPayload.parseFrom(spec.getPayload());
          String identifier = payload.getIdentifier();
          transformBuilder.putAnnotations(
              BeamUrns.getConstant(Annotations.Enum.SCHEMATRANSFORM_URN_KEY),
              ByteString.copyFromUtf8(identifier));
          if (identifier.equals(MANAGED_TRANSFORM_URN)) {
            Schema configSchema =
                SchemaTranslation.schemaFromProto(payload.getConfigurationSchema());
            Row configRow =
                RowCoder.of(configSchema).decode(payload.getConfigurationRow().newInput());
            String underlyingIdentifier = configRow.getString("transform_identifier");
            if (underlyingIdentifier == null) {
              throw new IllegalStateException(
                  String.format(
                      "Encountered a Managed Transform that has an empty \"transform_identifier\": %n%s",
                      configRow));
            }
            transformBuilder.putAnnotations(
                BeamUrns.getConstant(Annotations.Enum.MANAGED_UNDERLYING_TRANSFORM_URN_KEY),
                ByteString.copyFromUtf8(underlyingIdentifier));
          }
        }
      }

      Row configRow = null;
      try {
        configRow = payloadTranslator.toConfigRow(appliedPTransform.getTransform());
      } catch (UnsupportedOperationException e) {
        // Optional toConfigRow() has not been implemented. We can just ignore.
      } catch (Exception e) {
        LOG.warn(

View on GitHub (pinned to 12126d8942)