apache/druid · error · InterruptedException

Timed out waiting for operation

Error message

Timed out waiting for operation %s to complete

What it means

Raised after polling a GCE zone operation OPERATION_END_MAX_RETRIES times at POLL_INTERVAL_MS intervals without the operation reaching DONE status. The long-running GCE operation (e.g. instance start/terminate during autoscaling) is taking longer than the fixed retry budget allows, so the autoscaler gives up waiting and reports a timeout.

Solutions

1. Check the GCE console / `gcloud compute operations describe` for the operation's real status and any errors.
2. Increase OPERATION_END_MAX_RETRIES or POLL_INTERVAL_MS in GceAutoScaler if operations legitimately take long in your project.
3. Retry the autoscaling action; the operation may still complete asynchronously.
4. Check for quota exhaustion, capacity issues, or slow API responses in the target zone.

When it happens

Trigger: Thrown at extensions-contrib/gce-extensions/src/main/java/org/apache/druid/indexing/overlord/autoscaling/gce/GceAutoScaler.java:197 when the library encounters an invalid state.

Common situations: See trigger scenarios.

Understand the failure class

• Timeouts: ETIMEDOUT, deadlines, and hung requests — what actually expires when a request times out.

AI-assisted analysis of apache/druid@9b90983fd2 (2026-09-07). Data as JSON: /api/errors/a8ca068a0e42700a. Report an issue: GitHub.

Appendix: source

Thrown at extensions-contrib/gce-extensions/src/main/java/org/apache/druid/indexing/overlord/autoscaling/gce/GceAutoScaler.java:197

    String status = operation.getStatus();
    String opId = operation.getName();
    for (int i = 0; i < OPERATION_END_MAX_RETRIES; i++) {
      if (operation == null || "DONE".equals(status)) {
        return operation == null ? null : operation.getError();
      }
      log.info("Waiting for operation %s to end", opId);
      Thread.sleep(POLL_INTERVAL_MS);
      Compute.ZoneOperations.Get get = compute.zoneOperations().get(
          envConfig.getProjectId(),
          envConfig.getZoneName(),
          opId
      );
      operation = get.execute();
      if (operation != null) {
        status = operation.getStatus();
      }
    }
    throw new InterruptedException(
        StringUtils.format("Timed out waiting for operation %s to complete", opId)
    );
  }

  /**
   * When called resizes envConfig.getManagedInstanceGroupName() increasing it by creating
   * envConfig.getNumInstances() new workers (unless the maximum is reached). Return the
   * IDs of the workers created
   */
  @Override
  public AutoScalingData provision()
  {
    final String project = envConfig.getProjectId();
    final String zone = envConfig.getZoneName();
    final int numInstances = envConfig.getNumInstances();
    final String managedInstanceGroupName = envConfig.getManagedInstanceGroupName();

    try {

View on GitHub (pinned to 9b90983fd2)