apache/iceberg · error · FlinkRuntimeException

Failed to send operator %s coordinator global data statistic

Error message

Failed to send operator %s coordinator global data statistics to requesting subtask %d for checkpoint %d

What it means

When callInCoordinatorThread executes the callable directly on the coordinator thread (no executor hand-off), any Throwable thrown while sending global data statistics to a requesting subtask is logged and rethrown as a FlinkRuntimeException with the message 'Failed to send operator %s coordinator global data statistics to requesting subtask %d for checkpoint %d'. This converts coordinator-thread failures into job failures carrying operator, subtask, and checkpoint context.

Source

Thrown at flink/v1.20/flink/src/main/java/org/apache/iceberg/flink/sink/shuffle/DataStatisticsCoordinator.java:166

                    "Uncaught Exception in data statistics coordinator: {} executor",
                    operatorName,
                    t);
                ExceptionUtils.rethrowException(t);
                return null;
              }
            };

        coordinatorExecutor.submit(guardedCallable).get();
      } catch (InterruptedException | ExecutionException e) {
        throw new FlinkRuntimeException(errorMessage, e);
      }
    } else {
      try {
        callable.call();
      } catch (Throwable t) {
        LOG.error(
            "Uncaught Exception in data statistics coordinator: {} executor", operatorName, t);
        throw new FlinkRuntimeException(errorMessage, t);
      }
    }
  }

  public void runInCoordinatorThread(Runnable runnable) {
    this.coordinatorExecutor.execute(
        new ThrowableCatchingRunnable(
            throwable ->
                this.coordinatorThreadFactory.uncaughtException(Thread.currentThread(), throwable),
            runnable));
  }

  private void runInCoordinatorThread(ThrowingRunnable<Throwable> action, String actionString) {
    ensureStarted();
    runInCoordinatorThread(
        () -> {
          try {
            action.run();

View on GitHub (pinned to 86d9c8fc54)

Solutions

1. Read the wrapped cause ('Uncaught Exception in data statistics coordinator') for the real failure
2. Verify network/channel health between coordinator and subtasks; channel errors often surface here
3. Check payload size — very large sketches can fail serialization or allocation; reduce statistics cardinality
4. Upgrade or file an issue if the cause points to Iceberg code, with the full stack trace

try {
  coordinator.callInCoordinatorThread(callable, errorMessage, operatorName, subtaskId, checkpointId);
} catch (FlinkRuntimeException e) {
  Throwable root = ExceptionUtils.stripCompletionException(e.getCause());
  LOG.error("Global statistics send to subtask failed: {}", root.getMessage(), root);
  throw e;
}

When it happens

Trigger: A throwable escapes callable.call() executed inline on the coordinator thread while sending global statistics to a requesting subtask for a checkpoint; typically triggered via handleEventFromOperator processing a RequestGlobalStatisticsEvent.

Common situations: Failures during checkpoint-triggered statistics distribution: serialization of the sketch/statistics payload, downstream channel errors, or bugs in range-bound computation on the coordinator.

Related errors

• Failed to send operator %s coordinator global data statistic
• Unrecognized version or corrupt state: <version>
• Unknown serialize version: ${version}
• Unrecognized version or corrupt state: ${version}
• Failed to close equality delta writer

AI-assisted analysis of apache/iceberg@86d9c8fc54 (2026-09-12). Data as JSON: /api/errors/51eb8dbc8603c0ba. Report an issue: GitHub.