apache/hadoop · error · IOException

Mark not set

Error message

Mark not set

What it means

The companion guard in LimitInputStream.reset(): even on mark-capable streams it throws IOException("Mark not set") when mark(readLimit) was never called on this wrapper (the internal mark field is still -1). Marking is per-instance state, so re-wrapping the stream between mark and reset also loses the mark.

Source

Thrown at hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/util/LimitInputStream.java:95

    if (left == 0) {
      return -1;
    }

    len = (int) Math.min(len, left);
    int result = in.read(b, off, len);
    if (result != -1) {
      left -= result;
    }
    return result;
  }

  @Override
  public synchronized void reset() throws IOException {
    if (!in.markSupported()) {
      throw new IOException("Mark not supported");
    }
    if (mark == -1) {
      throw new IOException("Mark not set");
    }

    in.reset();
    left = mark;
  }

  @Override
  public long skip(long n) throws IOException {
    n = Math.min(n, left);
    long skipped = in.skip(n);
    left -= skipped;
    return skipped;
  }
}

View on GitHub (pinned to 2add963021)

Solutions

1. Call mark(readLimit) with a limit at least as large as the region you may re-read, immediately before reading it
2. Pair mark/reset inside one method so they cannot be separated by a refactor
3. Track a marked flag (or check the wrapper's state) before reset()

Example fix

// before
parser.maybeMark();
parser.consume();
limitIn.reset(); // throws if maybeMark skipped the mark

// after
limitIn.mark(MAX_LOOKAHEAD);
try {
  parser.consume();
} finally {
  limitIn.reset();
}

boolean marked = false;
if (in.markSupported()) { in.mark(MAX_LOOKAHEAD); marked = true; }
... // read
if (marked) { in.reset(); }

try { in.reset(); } catch (IOException e) { if ("Mark not set".equals(e.getMessage())) { /* mark was never made: re-read from a saved offset instead */ } else throw e; }

When it happens

Trigger: Calling reset() before any mark(); constructing a new LimitInputStream over the same source between the mark() and reset() calls; parser state machines where mark is conditional but reset is unconditional.

Common situations: Lookahead code refactored so the mark call moved into a branch; per-record utility methods that assume an outer layer already marked; resetting after the read budget (left) was exhausted by design.

Related errors

• Mark not supported
• Too many bytes before newline: " + bytesConsumed
• Too many bytes before delimiter: " + bytesConsumed
• %s: Stream is closed!
• Mark/reset not supported

AI-assisted analysis of apache/hadoop@2add963021 (2026-08-22). Data as JSON: /api/errors/aca6a8791de43cea. Report an issue: GitHub.