apache/beam · error · RuntimeException

Failed to validate transform %s

Error message

Failed to validate transform %s

What it means

Thrown as a RuntimeException wrapping any Exception raised by a per-URN validator in PipelineValidator.validateTransform. It indicates a transform in the pipeline proto failed its URN-specific validation (wrong coders, missing inputs/outputs, malformed subtransforms), and the original cause is attached as the supressed cause.

Source

Thrown at sdks/java/core/src/main/java/org/apache/beam/sdk/util/construction/graph/PipelineValidator.java:230

          "Transform %s references environment %s when no environment should be specified since it is a required runner implemented transform %s.",
          id,
          transform.getEnvironmentId(),
          urn);
    }

    if (Strings.isNullOrEmpty(urn)) {
      checkArgument(
          isComposite(transform),
          "Transform %s is not a composite transform but does not have a specified URN. %s",
          id,
          transform);
    }

    if (VALIDATORS.containsKey(urn)) {
      try {
        VALIDATORS.get(urn).validate(id, transform, components, requirements);
      } catch (Exception e) {
        throw new RuntimeException(String.format("Failed to validate transform %s", id), e);
      }
    }
  }

  private static void validateParDo(
      String id, PTransform transform, Components components, Set<String> requirements)
      throws Exception {
    ParDoPayload payload = ParDoPayload.parseFrom(transform.getSpec().getPayload());
    // side_inputs
    for (String sideInputId : payload.getSideInputsMap().keySet()) {
      checkArgument(
          transform.containsInputs(sideInputId),
          "Transform %s side input %s is not listed in the transform's inputs",
          id,
          sideInputId);
    }
    if (payload.getStateSpecsCount() > 0 || payload.getTimerFamilySpecsCount() > 0) {
      checkArgument(requirements.contains(ParDoTranslation.REQUIRES_STATEFUL_PROCESSING_URN));

View on GitHub (pinned to 12126d8942)

Solutions

1. Read the chained cause exception — it names the actual validation failure for the transform URN.
2. Fix the transform's inputs/outputs/coders per the cause (e.g. ensure required inputs are connected).
3. Regenerate the pipeline proto with the SDK instead of hand-editing it.
4. Run validation earlier in development (unit test using PipelineValidator) to catch construction bugs.

Example fix

// before: transform missing required input
PTransform.newBuilder().setUrn("beam:transform:flatten:v1")... // no inputs
// after
PTransform.newBuilder().setUrn("beam:transform:flatten:v1")
    .putInputs("in0", pcollId0).putInputs("in1", pcollId1)...

// pre-check common validator requirements before validation
for (PTransformNode t : pipeline.getTransforms()) {
  if (t.getTransform().getInputsCount() == 0 && REQUIRES_INPUT_URNS.contains(t.getTransform().getUrn())) throw new IllegalArgumentException("Transform " + t.getId() + " has no inputs");
}

boolean transformHasRequiredIO(RunnerApi.PTransform t) { return t.getInputsCount() > 0 && t.getOutputsCount() > 0; }

try { validatePipeline(pipeline); } catch (RuntimeException e) { Throwable cause = e.getCause(); log.error("Transform {} failed validation: {}", id, cause == null ? e : cause.getMessage(), cause); throw e; }

When it happens

Trigger: Validating a pipeline (validateComponents) where a PTransform with a registered URN fails VALIDATORS.get(urn).validate — e.g. missing required inputs, coder mismatches, unsupported payload structure.

Common situations: Hand-built pipeline protos; graph rewrites that broke transform contracts; runner-side validation of pipelines constructed by SDKs with bugs; inspect the cause to find the real problem.

Understand the failure class

Background: Schema validation failed / invalid input schema: payload rejected because its shape doesn't match the expected schema — this error's family across 28 libraries.

Related errors

• Pipeline update will not be possible because the following t
• Illegal access to pipeline after visitor traversal was compl
• One or more ErrorHandlers aren't closed, and this pipeline c
• Failed to validate %s
• Failed to validate %s

AI-assisted analysis of apache/beam@12126d8942 (2026-09-13). Data as JSON: /api/errors/7d50bbbb094465d6. Report an issue: GitHub.