apache/hadoop · error · IOException

Mark not supported

Error message

Mark not supported

What it means

LimitInputStream wraps an InputStream and enforces a remaining-byte budget. Its reset() is only legal when the wrapped stream itself can mark: it delegates the check to in.markSupported() and throws IOException("Mark not supported") when the underlying stream (FileInputStream, a raw socket stream, etc.) cannot mark/reset.

Source

Thrown at hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/util/LimitInputStream.java:92

    if (len == 0) {
      return 0;
    }
    if (left == 0) {
      return -1;
    }

    len = (int) Math.min(len, left);
    int result = in.read(b, off, len);
    if (result != -1) {
      left -= result;
    }
    return result;
  }

  @Override
  public synchronized void reset() throws IOException {
    if (!in.markSupported()) {
      throw new IOException("Mark not supported");
    }
    if (mark == -1) {
      throw new IOException("Mark not set");
    }

    in.reset();
    left = mark;
  }

  @Override
  public long skip(long n) throws IOException {
    n = Math.min(n, left);
    long skipped = in.skip(n);
    left -= skipped;
    return skipped;
  }
}

View on GitHub (pinned to 2add963021)

Solutions

1. Wrap the source in a BufferedInputStream before the LimitInputStream so mark/reset is supported
2. Check markSupported() before calling reset() and take a non-marking fallback path (re-open or skip-forward)
3. Restructure the parser to count bytes instead of marking, or use PushbackInputStream.unread()

Example fix

// before
InputStream in = new LimitInputStream(new FileInputStream(file), 4096);
in.mark(64); ... in.reset();

// after
InputStream in = new LimitInputStream(
    new BufferedInputStream(new FileInputStream(file)), 4096);
in.mark(64); ... in.reset();

InputStream src = new BufferedInputStream(new FileInputStream(file)); // marks supported
LimitInputStream in = new LimitInputStream(src, limit);
...
if (!in.markSupported()) { /* re-open stream instead of reset */ }

try { in.reset(); } catch (IOException e) { if ("Mark not supported".equals(e.getMessage())) { /* fall back: re-open source and skip consumed bytes */ } else throw e; }

When it happens

Trigger: new LimitInputStream(new FileInputStream(file), limit).reset(); wrapping any non-marking stream and calling mark()/reset(); a record parser that uses mark/reset for lookahead pointed at file or network streams.

Common situations: Swapping the underlying stream from BufferedInputStream (supports mark) to FileInputStream or a socket stream; code tested against ByteArrayInputStream (marks fine) but run on files; binary parsers relying on bounded lookahead.

Related errors

• Mark not set
• Too many bytes before newline: " + bytesConsumed
• Too many bytes before delimiter: " + bytesConsumed
• %s: Stream is closed!
• Mark/reset not supported

AI-assisted analysis of apache/hadoop@2add963021 (2026-08-22). Data as JSON: /api/errors/77697cca335d5997. Report an issue: GitHub.