apache/iceberg · error · RuntimeException

Failed while running parallel task

Error message

Failed while running parallel task

What it means

ParallelIterable wraps a checked (non-RuntimeException) Throwable thrown by one of its worker tasks in a RuntimeException with this message. Because ExecutionException loses the direct throw, the original cause is attached as the cause.

Solutions

1. Inspect e.getCause() for the real failure and fix it
2. Wrap checked exceptions in your task iterables into RuntimeExceptions if you want them rethrown directly
3. Increase task robustness (retries, validation) for parallel tasks

Example fix

// before
CloseableIterable<T> it = ParallelIterable.of(iterables, pool); // task throws IOException
// after
Iterable<T> safe = () -> new Iterator<T>() { /* wrap IOException in RuntimeException */ };

// validate task iterables don't throw checked exceptions
iterables.forEach(it -> Preconditions.checkNotNull(it, "task iterable must not be null"));

try { while (it.hasNext()) it.next(); } catch (RuntimeException e) { Throwable cause = e.getCause(); /* handle real cause */ }

When it happens

Trigger: Iterating a ParallelIterable (hasNext → checkTasks) while a taskFuture completes exceptionally with a checked Throwable, e.g. an IOException thrown inside a supplied Iterable's task.

Common situations: Parallel manifest/file scans where one worker hits an IO error or a checked library exception.

Understand the failure class

Background: "This is a bug, please report it": internal invariant violations, unreachable panics, and SNH errors explained — this error's family across 47 libraries.

Related errors

• Failed to check range end
• Failed to close task iterator
• Interrupted while running parallel task
• Unknown throwable

AI-assisted analysis of apache/iceberg@86d9c8fc54 (2026-09-12). Data as JSON: /api/errors/10fcf40acab666f5. Report an issue: GitHub.

Appendix: source

Thrown at core/src/main/java/org/apache/iceberg/util/ParallelIterable.java:164

     */
    private synchronized boolean checkTasks() {
      Preconditions.checkState(!closed.get(), "Already closed");
      boolean hasRunningTask = false;

      for (int i = 0; i < taskFutures.length; i += 1) {
        if (taskFutures[i] == null || taskFutures[i].isDone()) {
          if (taskFutures[i] != null) {
            // check for task failure and re-throw any exception. Enqueue continuation if any.
            try {
              Optional<Task<T>> continuation = taskFutures[i].get();
              continuation.ifPresent(yieldedTasks::addLast);
              taskFutures[i] = null;
            } catch (ExecutionException e) {
              if (e.getCause() instanceof RuntimeException) {
                // rethrow a runtime exception
                throw (RuntimeException) e.getCause();
              } else {
                throw new RuntimeException("Failed while running parallel task", e.getCause());
              }
            } catch (InterruptedException e) {
              throw new RuntimeException("Interrupted while running parallel task", e);
            }
          }

          // submit a new task if there is space in the queue
          if (queue.size() < maxQueueSize) {
            taskFutures[i] = submitNextTask();
          }
        }

        if (taskFutures[i] != null) {
          hasRunningTask = true;
        }
      }

      return !closed.get() && (tasks.hasNext() || hasRunningTask);

View on GitHub (pinned to 86d9c8fc54)