apache/flink · error · IOException

Input opening request timed out. Opener was {} alive. Stack

Error message

Input opening request timed out. Opener was {} alive. Stack of split open thread:
{}

What it means

Thrown by InputSplitOpenThread.waitForCompletion when the async file-open thread does not finish within the configured openTimeout. The message records whether the opener thread was still alive and dumps its stack trace, which is the primary diagnostic for identifying where the open is stuck.

Source

Thrown at flink-core/src/main/java/org/apache/flink/api/common/io/FileInputFormat.java:1043

            if (this.error != null) {
                throw this.error;
            }
            if (this.fdis != null) {
                return this.fdis;
            } else {
                // double-check that the stream has not been set by now. we don't know here whether
                // a) the opener thread recognized the canceling and closed the stream
                // b) the flag was set such that the stream did not see it and we have a valid
                // stream
                // In any case, close the stream and throw an exception.
                abortWait();

                final boolean stillAlive = this.isAlive();
                final StringBuilder bld = new StringBuilder(256);
                for (StackTraceElement e : this.getStackTrace()) {
                    bld.append("\tat ").append(e.toString()).append('\n');
                }
                throw new IOException(
                        "Input opening request timed out. Opener was "
                                + (stillAlive ? "" : "NOT ")
                                + " alive. Stack of split open thread:\n"
                                + bld.toString());
            }
        }

        /** Double checked procedure setting the abort flag and closing the stream. */
        private void abortWait() {
            this.aborted = true;
            final FSDataInputStream inStream = this.fdis;
            this.fdis = null;
            if (inStream != null) {
                try {
                    inStream.close();
                } catch (Throwable t) {
                }
            }

View on GitHub (pinned to 2f3c205e92)

Solutions

Raise openTimeout via the input format: format.setOpenTimeout(...) or the 'input.file.open.timeout' / CoreOptions config, sized above the store's worst-case open latency.
Use the stack trace in the message to identify the blocking call (e.g., stuck in Namenode RPC, S3 client) and address that layer.
For object stores, tune the store-specific client timeout/retry config and ensure proper connection pooling.
Verify network/DNS between TaskManagers and the storage endpoint; check for kerberos ticket expiry.

Example fix

// before: default openTimeout too short for S3
TextInputFormat format = new TextInputFormat(new Path("s3://bucket/data/"));

// after: raise the timeout for high-latency stores
format.setOpenTimeout(5 * 60 * 1000L); // 5 minutes

Defensive patterns

Strategy: retry

Validate before calling

// Size openTimeout to the store's worst-case open latency
long storeOpenP99Ms = 30_000L; // measure for your store
long openTimeout = Math.max(format.getOpenTimeout(), storeOpenP99Ms * 4);
format.setOpenTimeout(openTimeout);

Try / catch

// Distinguish timeout from hard open errors to decide retry
try {
    format.open(split);
} catch (IOException e) {
    if (e.getMessage().contains("timed out")) {
        // transient: backoff and retry, or raise openTimeout
    } else {
        throw e;
    }
}

Prevention

Set openTimeout above the store's measured p99 open latency (especially for S3/GCS).
Use the stack trace in the message to find the blocking call.
Tune the object-store client's own retry/timeout settings.
Check DNS/kerberos health between TaskManager and storage.

When it happens

Trigger: The FileSystem.open call blocks indefinitely (slow HDFS namenode, S3 throttling, DNS hang); openTimeout is set too low for a high-latency store; the opener thread deadlocks on FS initialization; network partition between TaskManager and the storage layer.

Common situations: Reading from S3/GCS/Azure Blob with default openTimeout while the store is throttled; first-read cold path on a kerberized HDFS where TGT acquisition is slow; misconfigured openTimeout below the store's typical open latency; TaskManager on a host with DNS resolution stalls.

Understand the failure class

Timeouts: ETIMEDOUT, deadlines, and hung requests — what actually expires when a request times out.

Related errors

AI-assisted analysis of apache/flink@2f3c205e92 (2026-08-14). Data as JSON: /api/errors/a20516a27f009e7b. Report an issue: GitHub.