apache/iceberg · error · FlinkRuntimeException

Failed to send operator %s coordinator global data statistic

Error message

Failed to send operator %s coordinator global data statistics for checkpoint %d

What it means

DataStatisticsCoordinator.callInCoordinatorThread runs a callable on the coordinator's single-threaded executor and waits for the result. When the callable that sends global data statistics for a checkpoint fails (the guardedFuture is completed exceptionally on the coordinator thread), the waiting thread wraps the failure in a FlinkRuntimeException with the formatted message 'Failed to send operator %s coordinator global data statistics for checkpoint %d'.

Source

Thrown at flink/v1.20/flink/src/main/java/org/apache/iceberg/flink/sink/shuffle/DataStatisticsCoordinator.java:158

    if (!coordinatorThreadFactory.isCurrentThreadCoordinatorThread()) {
      try {
        Callable<Void> guardedCallable =
            () -> {
              try {
                return callable.call();
              } catch (Throwable t) {
                LOG.error(
                    "Uncaught Exception in data statistics coordinator: {} executor",
                    operatorName,
                    t);
                ExceptionUtils.rethrowException(t);
                return null;
              }
            };

        coordinatorExecutor.submit(guardedCallable).get();
      } catch (InterruptedException | ExecutionException e) {
        throw new FlinkRuntimeException(errorMessage, e);
      }
    } else {
      try {
        callable.call();
      } catch (Throwable t) {
        LOG.error(
            "Uncaught Exception in data statistics coordinator: {} executor", operatorName, t);
        throw new FlinkRuntimeException(errorMessage, t);
      }
    }
  }

  public void runInCoordinatorThread(Runnable runnable) {
    this.coordinatorExecutor.execute(
        new ThrowableCatchingRunnable(
            throwable ->
                this.coordinatorThreadFactory.uncaughtException(Thread.currentThread(), throwable),
            runnable));

View on GitHub (pinned to 86d9c8fc54)

Solutions

1. Inspect the root cause in the exception chain (the cause of the FlinkRuntimeException) — the actual failure happened on the coordinator thread
2. Check JobManager memory and GC logs for OOM or long pauses during checkpoints
3. Reduce data statistics volume (larger checkpoint interval, fewer tracked keys) to lighten coordinator work
4. If the cause is a serialization/aggregation bug, upgrade Iceberg or report with the stack trace

try {
  coordinator.callInCoordinatorThread(callable, errorMessage, operatorName, checkpointId);
} catch (FlinkRuntimeException e) {
  LOG.error("Coordinator statistics send failed; cause: {}", ExceptionUtils.stringifyException(ExceptionUtils.findThrowable(e, Throwable.class).orElse(e)));
  throw e;
}

When it happens

Trigger: An exception occurs on the coordinator executor while sending global data statistics for a checkpoint, and another thread is blocked in waitForCoordinatorToProcessActions waiting for that action to complete; the ExecutionException is rethrown as FlinkRuntimeException.

Common situations: Coordinator-side failures during checkpoint alignment such as serialization errors of the accumulated statistics, OOM on the JobManager, or a bug in the statistics aggregation running on the coordinator executor.

Related errors

• Failed to send operator %s coordinator global data statistic
• Unrecognized version or corrupt state: <version>
• Unknown serialize version: ${version}
• Unrecognized version or corrupt state: ${version}
• Failed to close equality delta writer

AI-assisted analysis of apache/iceberg@86d9c8fc54 (2026-09-12). Data as JSON: /api/errors/ff9739217e9c0571. Report an issue: GitHub.