apache/hadoop · error · IOException

We encountered an error reading ${streams[curIdx].getName()}

Error message

We encountered an error reading ${streams[curIdx].getName()}.  During automatic edit log failover, we noticed that all of the remaining edit log streams are shorter than the current one!  The best remaining edit log ends at transaction ${newLast}, but we thought we could read up to transaction ${oldLast}.  If you continue, metadata will be lost forever!

What it means

Error "We encountered an error reading ${streams[curIdx].getName()}. During automatic edit log failover, we noticed that all of the remaining edit log streams are shorter than the current one! The best remaining edit log ends at transaction ${newLast}, but we thought we could read up to transaction ${oldLast}. If you continue, metadata will be lost forever!" thrown in apache/hadoop.

Source

Thrown at hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/RedundantEditLogInputStream.java:228

                  "at txid " + prevTxId + "; expected file to go up to " +
                  streams[curIdx].getLastTxId());
            }
          }
          prevTxId = op.getTransactionId();
          return op;
        } catch (IOException e) {
          prevException = e;
          state = State.STREAM_FAILED;
        }
        break;
      case STREAM_FAILED:
        if (curIdx + 1 == streams.length) {
          throw prevException;
        }
        long oldLast = streams[curIdx].getLastTxId();
        long newLast = streams[curIdx + 1].getLastTxId();
        if (newLast < oldLast) {
          throw new IOException("We encountered an error reading " +
              streams[curIdx].getName() + ".  During automatic edit log " +
              "failover, we noticed that all of the remaining edit log " +
              "streams are shorter than the current one!  The best " +
              "remaining edit log ends at transaction " +
              newLast + ", but we thought we could read up to transaction " +
              oldLast + ".  If you continue, metadata will be lost forever!",
              prevException);
        }
        LOG.error("Got error reading edit log input stream " +
          streams[curIdx].getName() + "; failing over to edit log " +
          streams[curIdx + 1].getName(), prevException);
        curIdx++;
        state = State.SKIP_UNTIL;
        break;
      case STREAM_FAILED_RESYNC:
        if (curIdx + 1 == streams.length) {
          if (prevException instanceof PrematureEOFException) {
            // bypass early EOF check

View on GitHub (pinned to 2add963021)

Solutions

1. Do not continue blindly: restore the missing edit log data from a backup or another JournalNode; continuing loses metadata permanently.

When it happens

Trigger: Automatic edit log failover finds all remaining streams shorter than the one being read.

Common situations: JournalNode quorum loss or disk failures truncating the edit log during NameNode startup recovery.

AI-assisted analysis of apache/hadoop@2add963021 (2026-08-22). Data as JSON: /api/errors/a87544271bd6edff. Report an issue: GitHub.