apache/hadoop · error · LogHeaderCorruptException

No header found in log

Error message

No header found in log

What it means

LogHeaderCorruptException('No header found in log') from EditLogFileInputStream.setup(): while opening an edit-log segment, the read of the layout-version header hit EOF before any bytes were consumed, meaning the segment file is zero-length (or truncated before the 4-byte version int). The NameNode treats such segments as corrupt; genuinely empty in-progress segments may be sidelined, but an unexpected empty segment aborts journal replay or validation.

Source

Thrown at hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/EditLogFileInputStream.java:164

    this.lastTxId = lastTxId;
    this.isInProgress = isInProgress;
    this.maxOpSize = DFSConfigKeys.DFS_NAMENODE_MAX_OP_SIZE_DEFAULT;
  }

  private void init(boolean verifyLayoutVersion)
      throws LogHeaderCorruptException, IOException {
    Preconditions.checkState(state == State.UNINIT);
    BufferedInputStream bin = null;
    InputStream fStream = null;
    try {
      fStream = log.getInputStream();
      bin = new BufferedInputStream(fStream);
      tracker = new FSEditLogLoader.PositionTrackingInputStream(bin);
      dataIn = new DataInputStream(tracker);
      try {
        logVersion = readLogVersion(dataIn, verifyLayoutVersion);
      } catch (EOFException eofe) {
        throw new LogHeaderCorruptException("No header found in log");
      }
      if (logVersion == -1) {
        // The edits in progress file is pre-allocated with 1MB of "-1" bytes
        // when it is created, then the header is written. If the header is
        // -1, it indicates the an exception occurred pre-allocating the file
        // and the header was never written. Therefore this is effectively a
        // corrupt and empty log.
        throw new LogHeaderCorruptException("No header present in log (value " +
            "is -1), probably due to disk space issues when it was created. " +
            "The log has no transactions and will be sidelined.");
      }
      // We assume future layout will also support ADD_LAYOUT_FLAGS
      if (NameNodeLayoutVersion.supports(
          LayoutVersion.Feature.ADD_LAYOUT_FLAGS, logVersion) ||
          logVersion < NameNodeLayoutVersion.CURRENT_LAYOUT_VERSION) {
        try {
          LayoutFlags.read(dataIn);
        } catch (EOFException eofe) {

View on GitHub (pinned to 2add963021)

Solutions

1. Restart with recovery: hdfs namenode -recover and choose to discard/sideline the corrupt segment (safe if it truly held no transactions).
2. If the file is edits_in_progress_* with no valid header, move it aside (it has no transactions) and restart from the previous fsimage plus earlier segments.
3. If real transactions are lost, restore the name dir from a checkpoint: bootstrapStandby from an HA peer, or -importCheckpoint from a SecondaryNameNode/backup copy.
4. Fix the durability root cause: keep dfs.namenode.name.dir on reliable local storage, check dmesg/RAID/fill levels, and stop copying live edits files.

Example fix

# before: NN startup fails with 'No header found in log' on edits_in_progress_0000000000012345
hdfs namenode -recover    # choose to discard the empty/corrupt segment
# after: NN starts, empty segment is sidelined, replay continues from prior segment

// peek the 4-byte layout version before handing a segment to any loader
try (DataInputStream in = new DataInputStream(
        new BufferedInputStream(Files.newInputStream(editsFile)))) {
  int v = in.readInt();   // EOFException here == 'No header found' case
  // v == -1 -> empty preallocated edits_in_progress; see error 2585
}

static boolean isCorruptLogHeader(IOException e) {
  return e instanceof EditLogFileInputStream.LogHeaderCorruptException;
}

try {
  EditLogFileInputStream in = new EditLogFileInputStream(file);
  in.refresh(false);
} catch (EditLogFileInputStream.LogHeaderCorruptException e) {
  // segment holds no readable transactions: sideline it (rename aside) and continue from prior segment
}

When it happens

Trigger: NameNode startup replay (FSEditLogLoader) or edit-log validation (EditLogFileInputStream.validateLog / offlineEditsViewer) opens an edits_* file that contains 0 bytes: a segment file created but never flushed (crash between open and header write), or a copy of live edits truncated by tooling.

Common situations: Power loss or disk-full at the moment a new edit segment was being rolled; name-dir on unreliable storage producing zero-length files; rsync/scp of a live edits file catching it mid-creation; monitoring scripts touching files in name/current.

Related errors

• EOF while reading layout flags from log
• Reached EOF when reading log header
• The log file {} seems to contain valid transactions ; journa
• Same delegation token being added twice; invalid entry in fs
• No header present in log (value is -1), probably due to disk

AI-assisted analysis of apache/hadoop@2add963021 (2026-08-22). Data as JSON: /api/errors/1f1a2089b11f5f79. Report an issue: GitHub.