apache/flink · error · IllegalArgumentException

Wrapped InputStream: cannot search backwards.

Error message

Wrapped InputStream: cannot search backwards.

What it means

Thrown by InputStreamFSInputWrapper.seek(desired) when desired < current position. The wrapper adapts a plain forward-only java.io.InputStream to Flink's seekable FSInputStream interface, but can only skip forward (via InputStream.skip). Backward seeks are impossible without re-opening the stream, so they are rejected.

Source

Thrown at flink-core/src/main/java/org/apache/flink/api/common/io/InputStreamFSInputWrapper.java:52

public class InputStreamFSInputWrapper extends FSDataInputStream {

    private final InputStream inStream;

    private long pos = 0;

    public InputStreamFSInputWrapper(InputStream inStream) {
        this.inStream = inStream;
    }

    @Override
    public void close() throws IOException {
        this.inStream.close();
    }

    @Override
    public void seek(long desired) throws IOException {
        if (desired < this.pos) {
            throw new IllegalArgumentException("Wrapped InputStream: cannot search backwards.");
        }

        while (this.pos < desired) {
            long numReadBytes = this.inStream.skip(desired - pos);
            if (numReadBytes == -1) {
                throw new EOFException("Unexpected EOF during forward seek.");
            }
            this.pos += numReadBytes;
        }
    }

    @Override
    public long getPos() throws IOException {
        return this.pos;
    }

    @Override
    public int read() throws IOException {

View on GitHub (pinned to 2f3c205e92)

Solutions

1. Use a true FSDataInputStream backed by a seekable file (e.g. via a FileSystem that supports random access) instead of InputStreamFSInputWrapper for any code that may seek backward.
2. If you only ever need forward seeks, ensure all seek targets are >= the current position.
3. Buffer the stream into a byte array / temp file and wrap that for full random access when backward seeks are required.

Example fix

// before: backward seek throws
FSDataInputStream in = new InputStreamFSInputWrapper(socketStream);
in.seek(in.getPos() - 10);
// after: buffer to a seekable file first
Path tmp = bufferToTempFile(socketStream);
FSDataInputStream in = fs.open(tmp);
in.seek(in.getPos() - 10);

if (desired < wrapper.getPos()) {
    throw new IllegalStateException(
        "Cannot seek backward on InputStreamFSInputWrapper: desired=" + desired
        + " pos=" + wrapper.getPos());
}
wrapper.seek(desired);

// Use a capability flag to pick the right stream type
boolean needsBackwardSeek = ...;
FSDataInputStream in = needsBackwardSeek
    ? openSeekableFile(fs, path)              // true random access
    : new InputStreamFSInputWrapper(stream);  // forward-only

try {
    wrapper.seek(desired);
} catch (IllegalArgumentException e) {
    if (e.getMessage().contains("cannot search backwards")) {
        // re-open the underlying stream from the start, then skip forward
        reopenAndSkipTo(desired);
        return;
    }
    throw e;
}

When it happens

Trigger: Code treats an InputStreamFSInputWrapper as a random-access FSDataInputStream and calls seek to a position earlier than getPos(). Common when a reader needs to rewind (e.g. to re-read a block, retry a record, or seek to a checkpoint) but the underlying source is a socket/stream rather than a file.

Common situations: Wrapping a non-seekable source (HTTP stream, socket, pipe) and passing it to code that assumes file-like random access; checkpoint/restart logic that seeks backward; format readers that re-read headers; tests using a ByteArrayInputStream wrapper that expect full seeking.

Related errors

• Unexpected EOF during forward seek.
• An I/O error occurred while creating temporary file to extra
• Failed to create parent(s) for given base dir: %s
• Compaction file not exist: {path}
• An error occurred while copying the file.

AI-assisted analysis of apache/flink@2f3c205e92 (2026-08-14). Data as JSON: /api/errors/f2b37c67412de20c. Report an issue: GitHub.