apache/beam · error · IllegalStateException

Could not find input with name %s for %s transform

Error message

Could not find input with name %s for %s transform

What it means

Thrown as an IllegalStateException by WriteFilesTranslation.getAdditionalInputs when a side input named in the WriteFiles transform's sideInputs map is not present among the transform's declared inputs (getInputsOrThrow threw IOException/NoSuchElementException). The translation expected every side-input key to have a corresponding PCollection input id resolvable via the rehydrated components.

Source

Thrown at sdks/java/core/src/main/java/org/apache/beam/sdk/util/construction/WriteFilesTranslation.java:257

    @Override
    public FunctionSpec migrate(SdkComponents components) throws IOException {
      return FunctionSpec.newBuilder()
          .setUrn(PTransformTranslation.WRITE_FILES_TRANSFORM_URN)
          .setPayload(payloadForWriteFilesLike(this, components).toByteString())
          .build();
    }

    @Override
    public Map<TupleTag<?>, PValue> getAdditionalInputs() {
      Map<TupleTag<?>, PValue> additionalInputs = new HashMap<>();
      for (Map.Entry<String, SideInput> sideInputEntry : payload.getSideInputsMap().entrySet()) {
        try {
          additionalInputs.put(
              new TupleTag<>(sideInputEntry.getKey()),
              rehydratedComponents.getPCollection(
                  protoTransform.getInputsOrThrow(sideInputEntry.getKey())));
        } catch (IOException exc) {
          throw new IllegalStateException(
              String.format(
                  "Could not find input with name %s for %s transform",
                  sideInputEntry.getKey(), WriteFiles.class.getSimpleName()));
        }
      }
      return additionalInputs;
    }

    @Override
    public FunctionSpec translateSink(SdkComponents newComponents) {
      // TODO: re-register the environment with the new components
      return payload.getSink();
    }

    @Override
    public Map<String, SideInput> translateSideInputs(SdkComponents components) {
      // TODO: re-register the PCollections and UDF environments
      return MoreObjects.firstNonNull(

View on GitHub (pinned to 12126d8942)

Solutions

1. Ensure every side-input key in the WriteFiles transform also appears in the transform's inputs map with a valid PCollection id.
2. Verify the referenced PCollection id exists in the Components' pcollections map.
3. If you are rewriting the graph, update side_inputs whenever you rename/remove inputs.
4. Validate the pipeline with PipelineValidator before translation to catch dangling references.

Example fix

// before: side_inputs key "side" missing from inputs map
// after: when building the PTransform
PTransform.newBuilder().putInputs("side", sidePcollId).putSideInputs("side", sideInputProto)...

for (String sideKey : protoTransform.getSideInputsMap().keySet()) {
  if (!protoTransform.containsInputs(sideKey)) throw new IllegalStateException("Side input " + sideKey + " missing from WriteFiles transform inputs");
  if (!components.containsPcollections(protoTransform.getInputsOrThrow(sideKey))) throw new IllegalStateException("PCollection not in components: " + protoTransform.getInputsOrThrow(sideKey));
}

boolean sideInputsResolvable(RunnerApi.PTransform t, RunnerApi.Components c) { return t.getSideInputsMap().keySet().stream().allMatch(k -> t.containsInputs(k) && c.containsPcollections(t.getInputsOrThrow(k))); }

try { inputs = WriteFilesTranslation.getAdditionalInputs(transformNode, components); } catch (IllegalStateException e) { log.error("Dangling side-input reference on WriteFiles transform", e); throw e; }

When it happens

Trigger: Translating a RunnerApi.PTransform for WriteFiles whose side_inputs reference input keys absent from protoTransform.getInputsMap(), or whose input PCollection id does not exist in components (rehydratedComponents.getPCollection fails).

Common situations: Hand-assembled pipeline protos in tests; graph rewriting passes that rename or drop inputs without updating side-input mappings; corrupted pipeline graphs from custom runners.

Understand the failure class

Background: 'Could not be found', 'does not exist', 'not found in database': the resource-not-found family when an ID, slug, key, or URI lookup comes back empty — this error's family across 20 libraries.

Related errors

• Unsupported type of %s: %s
• Illegal access to pipeline after visitor traversal was compl
• Pipeline update will not be possible because the following t
• Unrecognized value for stable unique names:
• One or more ErrorHandlers aren't closed, and this pipeline c

AI-assisted analysis of apache/beam@12126d8942 (2026-09-13). Data as JSON: /api/errors/11827f57566f4dbb. Report an issue: GitHub.