apache/hadoop · error · IOException

The journal edits cache is not enabled, which is a requireme

Error message

The journal edits cache is not enabled, which is a requirement to fetch journaled edits via RPC. Please enable it via dfs.ha.tail-edits.in-progress

What it means

Journal.getJournaledEdits throws this IOException when a client (the NameNode tailing in-progress edits) calls the getJournaledEdits RPC but the JournalNode was not configured with the in-memory edits cache. The cache is enabled only by setting dfs.ha.tail-edits.in-progress=true, whose default is false (DFS_HA_TAILEDITS_INPROGRESS_DEFAULT). The cache is a hard prerequisite because journaled edits are served straight from memory, not by reading edit log files off disk.

Source

Thrown at hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/qjournal/server/Journal.java:749

          break;
        }
      }
      if (log != null && log.isInProgress()) {
        logs.add(new RemoteEditLog(log.getStartTxId(),
            getHighestWrittenTxId(), true));
      }
    }

    return new RemoteEditLogManifest(logs, getCommittedTxnId());
  }

  /**
   * @see QJournalProtocol#getJournaledEdits(String, String, long, int)
   */
  public GetJournaledEditsResponseProto getJournaledEdits(long sinceTxId,
      int maxTxns) throws IOException {
    if (cache == null) {
      throw new IOException("The journal edits cache is not enabled, which " +
          "is a requirement to fetch journaled edits via RPC. Please enable " +
          "it via " + DFSConfigKeys.DFS_HA_TAILEDITS_INPROGRESS_KEY);
    }
    long highestTxId = getHighestWrittenTxId();
    if (sinceTxId == highestTxId + 1) {
      // Requested edits that don't exist yet, but this is expected,
      // because namenode always get the journaled edits with the sinceTxId
      // equal to image.getLastAppliedTxId() + 1. Short-circuiting the cache here
      // and returning a response with a count of 0.
      metrics.rpcEmptyResponses.incr();
      return GetJournaledEditsResponseProto.newBuilder().setTxnCount(0).build();
    } else if (sinceTxId > highestTxId + 1) {
      // Requested edits that don't exist yet and this is unexpected. Means that there is a lag
      // in this journal that does not contain some edits that should exist.
      // Throw one NewerTxnIdException to make namenode treat this response as an exception.
      // More detailed info please refer to: HDFS-16659 and HDFS-16771.
      metrics.rpcEmptyResponses.incr();
      throw new NewerTxnIdException(

View on GitHub (pinned to 2add963021)

Solutions

Set dfs.ha.tail-edits.in-progress=true in hdfs-site.xml on every JournalNode and restart them so the edits cache is built.
Ensure the NameNodes that will tail also set dfs.ha.tail-edits.in-progress=true — mixed old/new nodes otherwise produce partial behavior.
If you do not want in-progress tailing, disable the feature on the tailing NameNode so it stops issuing getJournaledEdits RPCs.
After restart, verify with a jmx/log check that the JournalNode reports rpc txns served (metrics txnsServedViaRpc) instead of throwing.

Example fix

<!-- before: JournalNode started with default -->
<property>
  <name>dfs.ha.tail-edits.in-progress</name>
  <value>false</value>
</property>
<!-- IOException: The journal edits cache is not enabled ... -->

<!-- after: enable the cache on ALL JournalNodes (and on the tailing NameNodes), then restart -->
<property>
  <name>dfs.ha.tail-edits.in-progress</name>
  <value>true</value>
</property>

Defensive patterns

Strategy: validation

Validate before calling

boolean cacheEnabled = conf.getBoolean(
    DFSConfigKeys.DFS_HA_TAILEDITS_INPROGRESS_KEY,
    DFSConfigKeys.DFS_HA_TAILEDITS_INPROGRESS_DEFAULT); // default false
if (!cacheEnabled) {
  // skip getJournaledEdits; fall back to manifest+file tailing
}

Try / catch

try {
  GetJournaledEditsResponseProto r = jn.getJournaledEdits(sinceTxId, maxTxns);
} catch (IOException e) {
  if (e.getMessage().contains("journal edits cache is not enabled")) {
    // config gap on the JournalNode: report and disable in-progress tailing locally
  } else { throw e; }
}

Prevention

Set dfs.ha.tail-edits.in-progress=true on every JournalNode (and tailing NameNodes) before enabling the feature.
Deploy the same hdfs-site.xml aliasmap/HA keys to JNs and NNs via config management.
Smoke-test one getJournaledEdits RPC after enabling, before relying on it in production.

When it happens

Trigger: QJournalProtocol.getJournaledEdits(sinceTxId, maxTxns) is invoked while Journal.cache == null, i.e., the JournalNode started with dfs.ha.tail-edits.in-progress unset/false. Happens the moment a StandbyNameNode with tail-edits.in-progress enabled tries to tail from a JournalNode that lacks the setting.

Common situations: Enabling in-progress tailing on the NameNodes but forgetting the JournalNodes (the key must be set on JNs for the cache, and on NNs for the tailer); upgrading Hadoop and adopting the feature only on part of the HA ensemble; asymmetric hdfs-site.xml pushed by config management.

Related errors

AI-assisted analysis of apache/hadoop@2add963021 (2026-08-22). Data as JSON: /api/errors/06022f9c8b0206c0. Report an issue: GitHub.