apache/flink · error · IOException

Error opening the Input Split {} [{},{}]: {}

Error message

Error opening the Input Split {} [{},{}]: {}

What it means

Wraps any Throwable raised while opening an input split in FileInputFormat.open(FileInputSplit). The split is opened in an async InputSplitOpenThread; if that thread throws (file missing, permission denied, decompression failure, FS error), the original error is rethrown as an IOException with the split path and [start,length] range and the underlying cause attached.

Source

Thrown at flink-core/src/main/java/org/apache/flink/api/common/io/FileInputFormat.java:818

            LOG.debug(
                    "Opening input split "
                            + fileSplit.getPath()
                            + " ["
                            + this.splitStart
                            + ","
                            + this.splitLength
                            + "]");
        }

        // open the split in an asynchronous thread
        final InputSplitOpenThread isot = new InputSplitOpenThread(fileSplit, this.openTimeout);
        isot.start();

        try {
            this.stream = isot.waitForCompletion();
            this.stream = decorateInputStream(this.stream, fileSplit);
        } catch (Throwable t) {
            throw new IOException(
                    "Error opening the Input Split "
                            + fileSplit.getPath()
                            + " ["
                            + splitStart
                            + ","
                            + splitLength
                            + "]: "
                            + t.getMessage(),
                    t);
        }

        // get FSDataInputStream
        if (this.splitStart != 0) {
            this.stream.seek(this.splitStart);
        }
    }

    /**

View on GitHub (pinned to 2f3c205e92)

Solutions

1. Inspect the nested cause (getCause()) in the thrown IOException — it carries the real reason (FileNotFoundException, AccessControlException, etc.).
2. Confirm the file still exists at the path and the split range is within file bounds before retrying.
3. For compressed inputs, ensure the compression codec jar (e.g., hadoop-gz) is on the classpath and the extension is registered.
4. For kerberized clusters, verify ticket/TGT lifetime covers the job; increase token renewal.
5. Retry the job once transient FS/network issues clear; for object stores check endpoint and credentials.

Example fix

// before: decompression codec missing → wrapped error with no clear cause
format.setFilePath("hdfs:///data/events.log.gz");

// after: add the codec dependency and confirm the cause
catch (IOException e) {
  Throwable root = e.getCause() != null ? e.getCause() : e;
  LOG.error("split open failed, root cause: {}", root);
}

// Pre-flight: confirm split files exist and are readable
Path p = split.getPath();
FileSystem fs = p.getFileSystem();
if (!fs.exists(p)) {
    throw new FileNotFoundException("Split path missing before open: " + p);
}

try {
    format.open(split);
} catch (IOException e) {
    Throwable root = e.getCause() != null ? e.getCause() : e;
    LOG.error("Failed to open split {} [{},{}]: {}", split.getPath(), split.getStart(), split.getLength(), root);
    throw e;
}

When it happens

Trigger: The file referenced by a FileInputSplit no longer exists or moved between split computation and open; read permissions are missing; the underlying FileSystem.open fails; decorateInputStream fails because the file's compression codec is not on the classpath; stream.seek(splitStart) fails for a non-zero offset.

Common situations: Files deleted/archived by an upstream pipeline mid-job; permissions/ACLs tightened between planning and execution; reading compressed (.gz/.bz2) files without the corresponding Hadoop compression dependency; kerberos ticket expiry on a secured HDFS cluster; transient S3/network errors during open.

Related errors

• Input opening request timed out. Opener was {} alive. Stack
• An IO error occurred while accessing the staging FileSystem.
• Number of input splits has to be at least 1.
• The given offset is not contained in the any block.
• Mkdirs failed to create {targetDir}

AI-assisted analysis of apache/flink@2f3c205e92 (2026-08-14). Data as JSON: /api/errors/f3a2d790cf01b5ed. Report an issue: GitHub.