apache/beam · error · java.lang.RuntimeException

expansion service error: %s

Error message

expansion service error: %s

What it means

External.expand() sends a transform to a Beam expansion service over gRPC. When the response carries a non-empty error string, the service rejected the expansion request and this RuntimeException wraps and re-raises that error message. It indicates the expansion service failed to expand the transform (e.g. unknown URN, missing dependencies, service-side exception).

Source

Thrown at sdks/java/core/src/main/java/org/apache/beam/sdk/util/construction/External.java:284

                          return components.registerCoder(kv.getValue());
                        } catch (IOException e) {
                          throw new RuntimeException(e);
                        }
                      })));
      RunnerApi.Components originalComponents = components.toComponents();
      ExpansionApi.ExpansionRequest request =
          requestBuilder
              .setComponents(originalComponents)
              .setTransform(ptransformBuilder.build())
              .setNamespace(getNamespace())
              .setPipelineOptions(PipelineOptionsTranslation.toProto(p.getOptions()))
              .build();

      ExpansionApi.ExpansionResponse response =
          clientFactory.getExpansionServiceClient(endpoint).expand(request);

      if (!Strings.isNullOrEmpty(response.getError())) {
        throw new RuntimeException(
            String.format("expansion service error: %s", response.getError()));
      }

      Map<String, RunnerApi.Environment> newEnvironmentsWithDependencies =
          response.getComponents().getEnvironmentsMap().entrySet().stream()
              .filter(
                  kv ->
                      !originalComponents.getEnvironmentsMap().containsKey(kv.getKey())
                          && kv.getValue().getDependenciesCount() != 0)
              .collect(Collectors.toMap(Map.Entry::getKey, Map.Entry::getValue));

      expandedComponents =
          response.getComponents().toBuilder()
              .putAllEnvironments(resolveArtifacts(newEnvironmentsWithDependencies, endpoint))
              .build();
      expandedTransform = response.getTransform();
      expandedRequirements = response.getRequirementsList();

View on GitHub (pinned to 12126d8942)

Solutions

1. Read the embedded response.getError() text in the exception message — it names the service-side cause; fix that specific issue first.
2. Verify the expansion service endpoint (host:port) and that the service is running and reachable.
3. Ensure the expansion service was started with the jar/classpath containing the requested transform.
4. Align SDK and expansion service versions; restart the expansion service to pick up new transforms.

Example fix

// before: wrong/missing expansion service
to.apply(External.of("my:transform:v1", payload, "localhost:0"));
// after: start a real service with the right classpath and use that endpoint
java -jar beam-sdks-java-expansion-service.jar --classpath deps.jar --port 4444
transform.apply(External.of("my:transform:v1", payload, "localhost:4444"));

// check endpoint reachability before expand
try (Socket s = new Socket(host, port)) { /* ok */ } catch (IOException e) { throw new IllegalStateException("expansion service unreachable: " + endpoint, e); }

boolean isEndpointSet(String endpoint) { return endpoint != null && endpoint.matches(".+\\d+") && endpoint.contains(":"); }

try { expanded = External.of(urn, payload, endpoint).expand(input); } catch (RuntimeException e) { if (e.getMessage().startsWith("expansion service error:")) { /* log embedded service error, fail pipeline with clear message */ } throw e; }

When it happens

Trigger: Calling External.expand() (or Beam.expansion of a cross-language transform) against an expansion service endpoint that returns ExpansionResponse.error, e.g. wrong service port, transform URN the service does not support, or missing jar/dependency on the service side.

Common situations: Misconfigured expansion service address in PipelineOptions; cross-language transform (e.g. Kafka/SQL) whose service version doesn't match the SDK; expansion service crashed or lacks the requested transform's classpath entries.

Related errors

• Received unknown SQL Dialect '%s'. Known dialects: %s
• Expansion request to transform service failed.
• Could not process the expansion request: + request
• expansion response error
• Could not find coder for URN " + urn

AI-assisted analysis of apache/beam@12126d8942 (2026-09-13). Data as JSON: /api/errors/e58b442bbd5817ca. Report an issue: GitHub.