apache/hadoop · error · IOException

IPC's epoch {} is not the current writer epoch {} ; journal

Error message

IPC's epoch {} is not the current writer epoch  {} ; journal id: {}

What it means

checkWriteRequest runs after checkRequest and adds a second rule: writes (journal, startLogSegment, finalizeLogSegment) must come from the current WRITER — the epoch that actually won newEpoch and set lastWriterEpoch. An epoch that merely passes the promise check but differs from lastWriterEpoch gets this IOException. It stops a non-writer (e.g., a standby or a fenced-but-reconnected client) from mutating the log.

Source

Thrown at hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/qjournal/server/Journal.java:518

        Server.getRemoteIp(), currentEpochIpcSerial, journalId);
    currentEpochIpcSerial = reqInfo.getIpcSerialNumber();

    if (reqInfo.hasCommittedTxId()) {
      Preconditions.checkArgument(
          reqInfo.getCommittedTxId() >= committedTxnId.get(),
          "Client trying to move committed txid backward from " +
          committedTxnId.get() + " to " + reqInfo.getCommittedTxId() +
              " ; journal id: " + journalId);
      
      committedTxnId.set(reqInfo.getCommittedTxId());
    }
  }
  
  private synchronized void checkWriteRequest(RequestInfo reqInfo) throws IOException {
    checkRequest(reqInfo);
    
    if (reqInfo.getEpoch() != lastWriterEpoch.get()) {
      throw new IOException("IPC's epoch " + reqInfo.getEpoch() +
          " is not the current writer epoch  " +
          lastWriterEpoch.get() + " ; journal id: " + journalId);
    }
  }
  
  public synchronized boolean isFormatted() {
    return storage.isFormatted();
  }

  private void checkFormatted() throws JournalNotFormattedException {
    if (!isFormatted()) {
      throw new JournalNotFormattedException("Journal " +
          storage.getSingularStorageDir() + " not formatted" +
          " ; journal id: " + journalId);
    }
  }

  /**

View on GitHub (pinned to 2add963021)

Solutions

Re-establish the writer session: call getJournalState, then newEpoch with a strictly higher epoch, before any write RPC — writes are only valid from the current writer.
Verify only one NN is attempting to write to this journal id; put the other in standby.
If writing from custom tooling, follow the full QJM session order (getJournalState -> newEpoch -> writes) rather than journaling directly.
Check JN logs for the concurrent newEpoch that changed lastWriterEpoch — it identifies the writer that displaced this client.

Defensive patterns

Strategy: try-catch

Validate before calling

// Only send write RPCs when your epoch IS the current writer epoch;
// establish that explicitly:
GetJournalStateResponseProto st = jn.getJournalState(jid);
long epoch = Math.max(clock.nowMs(), st.getLastPromisedEpoch() + 1);
NewEpochResponseProto resp = jn.newEpoch(jid, nsInfo, epoch);
// only after this succeeds may journal()/startLogSegment() be sent

Type guard

static boolean isWrongWriterEpoch(IOException ioe) {
  return ioe.getMessage() != null
      && ioe.getMessage().contains("is not the current writer epoch");
}

Try / catch

try {
  jn.startLogSegment(reqInfo, txid, epoch);
} catch (IOException ioe) {
  if (isWrongWriterEpoch(ioe)) {
    // session is invalid: re-run getJournalState+newEpoch or stop writing
    reestablishWriterSession();
  } else {
    throw ioe;
  }
}

Prevention

Follow the full QJM session order (getJournalState -> newEpoch -> writes) in any direct protocol client.
Never reuse a cached RequestInfo/epoch across an epoch change.
Ensure exactly one NN per nameservice attempts writes at any time.

When it happens

Trigger: A client sends a write RPC whose epoch equals or exceeds the promised epoch but is not the epoch that established writer status: a client that skipped newEpoch for the current epoch, a second NN started with '-upgrade' against the same JNs mid-session, or protocol-level tooling writing without completing the writer handshake.

Common situations: Direct QJournalProtocol users (test harnesses, custom tools) that reuse an old session or forget newEpoch after an epoch bump; standby NN mistakenly configured to write; an NN restarted with cached epoch state attempting writes.

Related errors

AI-assisted analysis of apache/hadoop@2add963021 (2026-08-22). Data as JSON: /api/errors/2131aacba0d5a260. Report an issue: GitHub.