apache/hadoop · warning · LogHeaderCorruptException

No header present in log (value is -1), probably due to disk

Error message

No header present in log (value is -1), probably due to disk space issues when it was created. The log has no transactions and will be sidelined.

What it means

LogHeaderCorruptException thrown when the first int read from an edit log is -1 (0xFFFFFFFF). edits_in_progress files are pre-allocated with 1MB of -1 bytes before the real header overwrites them; reading -1 means the header was never written (an exception, classically disk full, interrupted the pre-allocation), so the segment is effectively empty and is sidelined. This is benign by design: the log has no transactions.

Source

Thrown at hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/EditLogFileInputStream.java:172

    BufferedInputStream bin = null;
    InputStream fStream = null;
    try {
      fStream = log.getInputStream();
      bin = new BufferedInputStream(fStream);
      tracker = new FSEditLogLoader.PositionTrackingInputStream(bin);
      dataIn = new DataInputStream(tracker);
      try {
        logVersion = readLogVersion(dataIn, verifyLayoutVersion);
      } catch (EOFException eofe) {
        throw new LogHeaderCorruptException("No header found in log");
      }
      if (logVersion == -1) {
        // The edits in progress file is pre-allocated with 1MB of "-1" bytes
        // when it is created, then the header is written. If the header is
        // -1, it indicates the an exception occurred pre-allocating the file
        // and the header was never written. Therefore this is effectively a
        // corrupt and empty log.
        throw new LogHeaderCorruptException("No header present in log (value " +
            "is -1), probably due to disk space issues when it was created. " +
            "The log has no transactions and will be sidelined.");
      }
      // We assume future layout will also support ADD_LAYOUT_FLAGS
      if (NameNodeLayoutVersion.supports(
          LayoutVersion.Feature.ADD_LAYOUT_FLAGS, logVersion) ||
          logVersion < NameNodeLayoutVersion.CURRENT_LAYOUT_VERSION) {
        try {
          LayoutFlags.read(dataIn);
        } catch (EOFException eofe) {
          throw new LogHeaderCorruptException("EOF while reading layout " +
              "flags from log");
        }
      }
      reader = FSEditLogOp.Reader.create(dataIn, tracker, logVersion);
      reader.setMaxOpSize(maxOpSize);
      state = State.OPEN;
    } finally {

View on GitHub (pinned to 2add963021)

Solutions

Free space on the name/edit device, then restart (or run hdfs namenode -recover): the NN sidelines the empty segment automatically.
If startup still loops, move the offending edits_in_progress_* aside; it contains no transactions.
Long term: alert on name-dir usage and move dfs.namenode.edits.dir (or QJM) off the full device.

Example fix

# before: 'No header present in log (value is -1)' at NN start after disk-full event
df -h /dfs/nn && rm -f /dfs/nn/current/edits_in_progress_*unused*
hdfs namenode -recover
# after: empty segment sidelined, NN starts clean

Defensive patterns

Strategy: try-catch

Validate before calling

int v = new DataInputStream(new BufferedInputStream(
    Files.newInputStream(editsInProgress))).readInt();
if (v == -1) {
  // preallocated-but-never-written header: segment is empty, safe to sideline
}

Try / catch

catch (EditLogFileInputStream.LogHeaderCorruptException e) {
  if (e.getMessage().contains("value is -1")) {
    // empty preallocated segment: free name-dir disk space and sideline; no transactions exist
  } else { throw e; }
}

Prevention

Alert on name/edit directory usage well before full -- the -1 header is the classic disk-full-at-roll signature.
Put dfs.namenode.edits.dir (or QJM) on a different device from fsimage storage.
Trust the NN's sidelining of empty segments; do not preemptively delete other segments.

When it happens

Trigger: Loading or validating an edits_in_progress_* segment whose creation failed after pre-allocation: disk full on the name/edit device at roll time, or a crash between preallocate and header write.

Common situations: Name-directory disk filling up (fsimage + retained edits); NN crash exactly during segment roll; repeated rolls failing on a full device leave several such files.

Related errors

AI-assisted analysis of apache/hadoop@2add963021 (2026-08-22). Data as JSON: /api/errors/ca3adc0278fd9b9a. Report an issue: GitHub.