apache/beam · error · TimeoutException

Transform Service did not start in

Error message

Transform Service did not start in {timeout / 1000} seconds.

What it means

waitTillUp polls the transform service logs (via docker compose / container status) until it sees startup-completion markers. If no matching logs appear within the given timeout, it throws this TimeoutException. It means the container was launched but never signaled readiness within the allotted window.

Solutions

Check container logs (docker compose logs / service status) for the actual startup failure (image pull error, port conflict, crash).
Increase the startup timeout argument to waitTillUp to accommodate slow image pulls.
Verify Docker/Docker Compose is installed, running, and the configured port is free before starting.
Retry the "up" command after fixing the underlying container failure.

Example fix

// before
service.waitTillUp(30_000); // too short for first image pull

// after
service.waitTillUp(300_000); // allow 5 minutes for image download

Defensive patterns

Strategy: try-catch

Validate before calling

// pre-check environment before calling waitTillUp
Process p = new ProcessBuilder("docker", "info").inheritIO().start();
if (p.waitFor() != 0) throw new IllegalStateException("Docker daemon not reachable");

Try / catch

try {
  service.waitTillUp(timeoutMs);
} catch (java.util.concurrent.TimeoutException e) {
  LOG.error("Transform service not ready: {}. Check container logs (docker compose logs)", e.getMessage());
  service.shutdown(); // clean up partial start
  throw e;
}

Prevention

Pre-pull the transform-service image before first launch to avoid slow startup.
Use generous timeouts (minutes, not seconds) for the first run.
Ensure the configured port is free before starting.

When it happens

Trigger: Calling waitTillUp (directly or via expand/upgradeTransformsViaTransformService/main with command "up") when the service containers fail to reach the ready log line before timeout: image pull failures, port conflicts, slow image pulls on first run, or the service crashing during startup. Note waitTillUp(-1) passes a negative/very large timeout.

Common situations: First-time start pulling a large Beam SDK transform-service image over a slow network; Docker daemon misconfiguration; the configured port already in use causing the service to exit; insufficient memory in the container environment.

Understand the failure class

Background: Request timed out: what client-side request timeouts mean across libraries (Request timed out, TIMED_OUT, APITimeoutError) — this error's family across 39 libraries.

Timeouts: ETIMEDOUT, deadlines, and hung requests — what actually expires when a request times out.

Related errors

AI-assisted analysis of apache/beam@12126d8942 (2026-09-13). Data as JSON: /api/errors/e5168e8851e35ed4. Report an issue: GitHub.

Appendix: source

Thrown at sdks/java/transform-service/launcher/src/main/java/org/apache/beam/sdk/transformservice/launcher/TransformServiceLauncher.java:316

    while (System.currentTimeMillis() - startTime < timeout) {
      String statusFileName = getStatus();
      try {
        // We are just waiting for a local process. No need for exponential backoff.
        this.wait(1000);
      } catch (InterruptedException e) {
        // Ignore and retry.
      }

      String output = String.join(" ", java.nio.file.Files.readAllLines(Paths.get(statusFileName)));
      if (!output.isEmpty()) {
        if (output.contains("transform-service")) {
          // Transform service was started since we found matching logs.
          return;
        }
      }
    }

    throw new TimeoutException(
        "Transform Service did not start in " + timeout / 1000 + " seconds.");
  }

  private synchronized String getStatus() throws IOException {
    File outputOverride = File.createTempFile("output_override", null);
    outputOverride.deleteOnExit();
    runDockerComposeCommand(ImmutableList.of("ps"), outputOverride);

    return outputOverride.getAbsolutePath();
  }

  private static class ArgConfig {

    static final String PROJECT_NAME_ARG_NAME = "project_name";
    static final String COMMAND_ARG_NAME = "command";
    static final String PORT_ARG_NAME = "port";
    static final String BEAM_VERSION_ARG_NAME = "beam_version";

View on GitHub (pinned to 12126d8942)