apache/hadoop · error · IOException

Tried to skip {} bytes past the limit at offset {}

Error message

Tried to skip {} bytes past the limit at offset {}

What it means

The skip() counterpart of the read limiter in PositionTrackingInputStream: skipping amt bytes would cross the limit that FSEditLogOp.Reader set via setLimit(maxOpSize) before decoding an op. In practice the reader skips the remainder of a frame whose declared length already exceeded the bound, so this marks the same condition as the read variant: garbage or torn data pushing a single op past its size limit.

Source

Thrown at hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/FSEditLogLoader.java:1418

    @Override
    public void reset() throws IOException {
      if (markPos == -1) {
        throw new IOException("Not marked!");
      }
      super.reset();
      curPos = markPos;
      markPos = -1;
    }

    public long getPos() {
      return curPos;
    }
    
    @Override
    public long skip(long amt) throws IOException {
      long extra = (curPos + amt) - limitPos;
      if (extra > 0) {
        throw new IOException("Tried to skip " + extra + " bytes past " +
            "the limit at offset " + limitPos);
      }
      long ret = super.skip(amt);
      curPos += ret;
      return ret;
    }
  }

  public long getLastAppliedTxId() {
    return lastAppliedTxId;
  }

  /**
   * Creates a Step used for updating startup progress, populated with
   * information from the given edits.  The step always includes the log's name.
   * If the log has a known length, then the length is included in the step too.
   * 
   * @param edits EditLogInputStream to use for populating step

View on GitHub (pinned to 2add963021)

Solutions

1. Treat the segment as corrupt: validate with 'hdfs oev' and recover with 'hdfs namenode -recover'
2. Raise dfs.namenode.max.op.size only when valid ops genuinely exceed the 50MB default
3. Restore a consistent fsimage plus edits from backup

Example fix

# before: scan/replay trips the skip limiter on a bad frame
# IOException: Tried to skip N bytes past the limit at offset L

# after: identify and skip the corrupt tail
hdfs oev -i <segment> -o /tmp/check.xml -p xml
hdfs namenode -recover

hdfs oev -i <segment> -o /tmp/check.xml -p xml   # validates op sizes and checksums before replay

try {
  op = in.readOp();
} catch (IOException ioe) {          // includes 'Tried to skip ... past the limit'
  LOG.warn("Bad frame at offset " + in.getPosition(), ioe);
  in.resync();
}

When it happens

Trigger: The op reader skipping past a corrupt op frame whose declared length exceeds maxOpSize during edit log load or scan; torn final segment after a NameNode crash.

Common situations: Corrupt segment tails; partial segment copies; misconfigured dfs.namenode.max.op.size.

Related errors

• Tried to read {} byte(s) past the limit at offset {}
• Error replaying edit log at offset {}. Expected transaction
• The layout version {} supports inodeId but gave bogus inodeI
• Invalid operation read {}
• Mismatched block IDs or generation stamps for the old last b

AI-assisted analysis of apache/hadoop@2add963021 (2026-08-22). Data as JSON: /api/errors/72b4eedd05ffdf87. Report an issue: GitHub.