apache/hadoop · error · LogHeaderCorruptException

EOF while reading layout flags from log

Error message

EOF while reading layout flags from log

What it means

LogHeaderCorruptException('EOF while reading layout flags from log') from EditLogFileInputStream.setup(): the 4-byte layout version was read successfully (and supports ADD_LAYOUT_FLAGS or comes from a newer layout), but LayoutFlags.read(dataIn) hit EOF. The header was only half written -- version present, layout-flags byte(s) missing -- so the segment is corrupt exactly like a missing header.

Source

Thrown at hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/EditLogFileInputStream.java:183

      }
      if (logVersion == -1) {
        // The edits in progress file is pre-allocated with 1MB of "-1" bytes
        // when it is created, then the header is written. If the header is
        // -1, it indicates the an exception occurred pre-allocating the file
        // and the header was never written. Therefore this is effectively a
        // corrupt and empty log.
        throw new LogHeaderCorruptException("No header present in log (value " +
            "is -1), probably due to disk space issues when it was created. " +
            "The log has no transactions and will be sidelined.");
      }
      // We assume future layout will also support ADD_LAYOUT_FLAGS
      if (NameNodeLayoutVersion.supports(
          LayoutVersion.Feature.ADD_LAYOUT_FLAGS, logVersion) ||
          logVersion < NameNodeLayoutVersion.CURRENT_LAYOUT_VERSION) {
        try {
          LayoutFlags.read(dataIn);
        } catch (EOFException eofe) {
          throw new LogHeaderCorruptException("EOF while reading layout " +
              "flags from log");
        }
      }
      reader = FSEditLogOp.Reader.create(dataIn, tracker, logVersion);
      reader.setMaxOpSize(maxOpSize);
      state = State.OPEN;
    } finally {
      if (reader == null) {
        IOUtils.cleanupWithLogger(LOG, dataIn, tracker, bin, fStream);
        state = State.CLOSED;
      }
    }
  }

  @Override
  public long getFirstTxId() {
    return firstTxId;
  }

View on GitHub (pinned to 2add963021)

Solutions

1. Run hdfs namenode -recover and discard/sideline the corrupt segment.
2. If the segment carried real transactions, restore the journal from a peer: bootstrapStandby (HA) or -importCheckpoint, or replay from the last good fsimage plus surviving segments.
3. Repair the storage/durability cause so headers cannot be torn again (local reliable disk for name dirs, QJM for shared edits).

Example fix

# before: 'EOF while reading layout flags from log' on startup
hdfs namenode -recover
# after: corrupt half-written segment sidelined; if transactions were lost,
# bootstrapStandby / -importCheckpoint from a peer's image

try (DataInputStream in = new DataInputStream(
        new BufferedInputStream(Files.newInputStream(editsFile)))) {
  int v = in.readInt();
  if (supportsLayoutFlags(v)) LayoutFlags.read(in);  // EOF here == torn header
}

catch (EditLogFileInputStream.LogHeaderCorruptException e) {
  if (e.getMessage().contains("layout flags")) {
    // torn header (version written, flags missing): segment is corrupt; sideline + fall back to last good image
  } else { throw e; }
}

When it happens

Trigger: A torn header write on segment creation (crash/disk-full after the version int but before layout flags); a truncated copy of a valid segment cut right after the version int; validated during NN startup replay, edits validation, or offlineEditsViewer.

Common situations: Same family as other torn edit-log headers: power loss during segment roll, full name device, or interrupted copying of journal files between hosts.

Related errors

• No header found in log
• Reached EOF when reading log header
• The log file {} seems to contain valid transactions ; journa
• Same delegation token being added twice; invalid entry in fs
• No header present in log (value is -1), probably due to disk

AI-assisted analysis of apache/hadoop@2add963021 (2026-08-22). Data as JSON: /api/errors/e17e785e1043328d. Report an issue: GitHub.