What causes "expansion service error: %s" in apache/beam?

Read the embedded response error text in the message — it contains the service's actual failure reason

apache/beam · error · RuntimeException

expansion service error: %s

Error message

expansion service error: %s

What it means

After sending an expansion request to the transform service, TransformUpgrader propagates any error reported in the ExpansionResponse by wrapping it in a RuntimeException with the 'expansion service error:' prefix. This surfaces server-side failures (bad config row, unknown URN, internal service error) to the pipeline author.

Source

Thrown at sdks/java/core/src/main/java/org/apache/beam/sdk/util/construction/TransformUpgrader.java:266

      // TransformService uses a compatible schema.
      optionsClone
          .as(StreamingOptions.class)
          .setUpdateCompatibilityVersion(ReleaseInfo.getReleaseInfo().getSdkVersion());
    }
    ExpansionApi.ExpansionRequest request =
        requestBuilder
            .setComponents(runnerAPIpipeline.getComponents())
            .setTransform(ptransformBuilder.build())
            .setNamespace(UPGRADE_NAMESPACE)
            .setPipelineOptions(PipelineOptionsTranslation.toProto(optionsClone))
            .addAllRequirements(runnerAPIpipeline.getRequirementsList())
            .build();

    ExpansionApi.ExpansionResponse response =
        clientFactory.getExpansionServiceClient(transformServiceEndpoint).expand(request);

    if (!Strings.isNullOrEmpty(response.getError())) {
      throw new RuntimeException(String.format("expansion service error: %s", response.getError()));
    }

    Map<String, RunnerApi.Environment> newEnvironmentsWithDependencies =
        response.getComponents().getEnvironmentsMap().entrySet().stream()
            .filter(
                kv ->
                    !runnerAPIpipeline.getComponents().getEnvironmentsMap().containsKey(kv.getKey())
                        && kv.getValue().getDependenciesCount() != 0)
            .collect(Collectors.toMap(Map.Entry::getKey, Map.Entry::getValue));

    RunnerApi.Components expandedComponents =
        response.getComponents().toBuilder()
            .putAllEnvironments(
                External.ExpandableTransform.resolveArtifacts(
                    newEnvironmentsWithDependencies, transformServiceEndpoint))
            .build();
    RunnerApi.PTransform expandedTransform = response.getTransform();

View on GitHub (pinned to 12126d8942)

Solutions

Read the embedded response error text in the message — it contains the service's actual failure reason
Verify the transform URN and config row schema match what the service supports
Run the transform service with a Beam version matching your SDK, and check the service logs

Example fix

// before
// request built with wrong config row schema
Row config = Row.withSchema(wrongSchema).addValues(...).build();
// after
Row config = Row.withSchema(serviceSupportedSchema).addValues(...).build(); // matches service expectations

Defensive patterns

Strategy: retry

Validate before calling

// Pre-validate the config row schema against the transform's advertised schema before expansion:
Schema expected = discoverTransformSchema(urn, serviceAddress); // via expansion service discovery
if (!config.getSchema().equals(expected)) {
  throw new IllegalStateException("Config row schema mismatch for URN " + urn);
}

Try / catch

try { pipeline.run(); } catch (RuntimeException e) {
  if (e.getMessage() != null && e.getMessage().startsWith("expansion service error:")) {
    log.error("Transform service failed: {}", e.getMessage());
    // fix URN/config or restart matching-version service, then retry
  } else throw e;
}

Prevention

Read the embedded service error — it names the root cause
Match transform service Beam version with your SDK version
Validate schema-transform URNs and config rows before submission
Check transform service logs for the server-side stack trace

When it happens

Trigger: The transform service fails to expand/upgrade a transform: unknown schema-transform URN, invalid configuration row, incompatible Beam version, or an internal exception inside the service.

Common situations: Wrong URN or malformed config row for a schema transform; transform service of a different Beam version than the SDK; the service-side transform throwing during expansion.

Related errors

AI-assisted analysis of apache/beam@12126d8942 (2026-09-13). Data as JSON: /api/errors/780e74f011d166df. Report an issue: GitHub.