Kudu client executor did not terminate after forced…

Error message

Kudu client executor did not terminate after forced shutdown.

What it means

WARN logged by KuduClientResource.awaitExecutorTermination when the executor still has not terminated after shutdownNow() and the forced-shutdown wait — tasks ignored interruption; resources may leak but close() proceeds.

Solutions

Capture a thread dump at shutdown to find the stuck threads and their stack traces
Check network reachability to Kudu masters/tablet servers from the task nodes
Fix or upgrade code performing blocking I/O on the executor (use timeouts at call sites)
If persistent, consider restarting the TaskManager/node to reclaim leaked threads

Defensive patterns

Strategy: try-catch

Validate before calling

// Detect stuck I/O early: bound Kudu RPCs with timeouts
KuduClient client = new KuduClient.KuduClientBuilder(master)
    .defaultOperationTimeoutMs(30_000)
    .defaultAdminOperationTimeoutMs(30_000)
    .build();

Try / catch

try {
    executor.shutdownNow();
    if (!executor.awaitTermination(FORCED_SECONDS, TimeUnit.SECONDS)) {
        // threads are stuck ignoring interrupts: trigger thread-dump/alerting hook
    }
} catch (InterruptedException ie) {
    executor.shutdownNow();
    Thread.currentThread().interrupt();
}

Prevention

Keep Kudu masters/tablet servers reachable — non-interruptible I/O is the main cause
Use operation timeouts on all Kudu calls so tasks can't block forever
Capture thread dumps when this warning fires to identify leaking code paths

When it happens

Trigger: close() → awaitExecutorTermination: shutdownNow() was issued but the second awaitTermination(FORCED_SHUTDOWN_TIMEOUT_SECONDS) still returns false, so this warning is logged.

Common situations: Threads blocked in non-interruptible I/O to unreachable Kudu masters/tablets; hung DNS resolution; native code or locks ignoring interrupts; leaked tasks holding resources.

Related errors

AI-assisted analysis of apache/seatunnel@cf67b549a7 (2026-09-10). Data as JSON: /api/errors/2b523522a2babd18. Report an issue: GitHub.

Appendix: source

Thrown at seatunnel-connectors-v2/connector-kudu/src/main/java/org/apache/seatunnel/connectors/seatunnel/kudu/kuduclient/KuduClientResource.java:74

            kuduClient.close();
        } finally {
            awaitExecutorTermination();
        }
    }

    private void awaitExecutorTermination() {
        executorService.shutdown();
        try {
            if (executorService.awaitTermination(
                    GRACEFUL_SHUTDOWN_TIMEOUT_SECONDS, TimeUnit.SECONDS)) {
                return;
            }

            log.warn("Kudu client executor did not terminate gracefully, forcing shutdown.");
            executorService.shutdownNow();
            if (!executorService.awaitTermination(
                    FORCED_SHUTDOWN_TIMEOUT_SECONDS, TimeUnit.SECONDS)) {
                log.warn("Kudu client executor did not terminate after forced shutdown.");
            }
        } catch (InterruptedException e) {
            executorService.shutdownNow();
            Thread.currentThread().interrupt();
            log.warn("Interrupted while waiting for Kudu client executor to terminate.", e);
        }
    }
}

View on GitHub (pinned to cf67b549a7)