{"record":{"id":"a87544271bd6edff","repo":"apache/hadoop","slug":"we-encountered-an-error-reading-streams-curidx","errorCode":null,"errorMessage":"We encountered an error reading ${streams[curIdx].getName()}.  During automatic edit log failover, we noticed that all of the remaining edit log streams are shorter than the current one!  The best remaining edit log ends at transaction ${newLast}, but we thought we could read up to transaction ${oldLast}.  If you continue, metadata will be lost forever!","messagePattern":"We encountered an error reading (.+?)\\.  During automatic edit log failover, we noticed that all of the remaining edit log streams are shorter than the current one!  The best remaining edit log ends at transaction (.+?), but we thought we could read up to transaction (.+?)\\.  If you continue, metadata will be lost forever!","errorType":"exception","errorClass":"IOException","httpStatus":null,"severity":"error","filePath":"hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/RedundantEditLogInputStream.java","lineNumber":228,"sourceCode":"                  \"at txid \" + prevTxId + \"; expected file to go up to \" +\n                  streams[curIdx].getLastTxId());\n            }\n          }\n          prevTxId = op.getTransactionId();\n          return op;\n        } catch (IOException e) {\n          prevException = e;\n          state = State.STREAM_FAILED;\n        }\n        break;\n      case STREAM_FAILED:\n        if (curIdx + 1 == streams.length) {\n          throw prevException;\n        }\n        long oldLast = streams[curIdx].getLastTxId();\n        long newLast = streams[curIdx + 1].getLastTxId();\n        if (newLast < oldLast) {\n          throw new IOException(\"We encountered an error reading \" +\n              streams[curIdx].getName() + \".  During automatic edit log \" +\n              \"failover, we noticed that all of the remaining edit log \" +\n              \"streams are shorter than the current one!  The best \" +\n              \"remaining edit log ends at transaction \" +\n              newLast + \", but we thought we could read up to transaction \" +\n              oldLast + \".  If you continue, metadata will be lost forever!\",\n              prevException);\n        }\n        LOG.error(\"Got error reading edit log input stream \" +\n          streams[curIdx].getName() + \"; failing over to edit log \" +\n          streams[curIdx + 1].getName(), prevException);\n        curIdx++;\n        state = State.SKIP_UNTIL;\n        break;\n      case STREAM_FAILED_RESYNC:\n        if (curIdx + 1 == streams.length) {\n          if (prevException instanceof PrematureEOFException) {\n            // bypass early EOF check","sourceCodeStart":210,"sourceCodeEnd":246,"githubUrl":"https://github.com/apache/hadoop/blob/2add9630210752f88ceb1bb74eb65e37bf41da8e/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/RedundantEditLogInputStream.java#L210-L246","documentation":"Error \"We encountered an error reading ${streams[curIdx].getName()}.  During automatic edit log failover, we noticed that all of the remaining edit log streams are shorter than the current one!  The best remaining edit log ends at transaction ${newLast}, but we thought we could read up to transaction ${oldLast}.  If you continue, metadata will be lost forever!\" thrown in apache/hadoop.","triggerScenarios":"Automatic edit log failover finds all remaining streams shorter than the one being read.","commonSituations":"JournalNode quorum loss or disk failures truncating the edit log during NameNode startup recovery.","solutions":["Do not continue blindly: restore the missing edit log data from a backup or another JournalNode; continuing loses metadata permanently."],"exampleFix":null,"handlingStrategy":null,"validationCode":null,"typeGuard":null,"tryCatchPattern":null,"preventionTips":[],"tags":[],"backgroundTag":null,"analyzedSha":"2add9630210752f88ceb1bb74eb65e37bf41da8e","analyzedAt":"2026-08-22T19:55:07.957Z","schemaVersion":2},"datasetVersion":"2026-08-23T01:17:44.959Z"}