{"record":{"id":"272c00343fdc0fea","repo":"apache/hadoop","slug":"cannot-obtain-block-length-for-of","errorCode":null,"errorMessage":"Cannot obtain block length for {} of {}","messagePattern":"Cannot obtain block length for (.+?) of (.+?)","errorType":"exception","errorClass":"CannotObtainBlockLengthException","httpStatus":null,"severity":"error","filePath":"hadoop-hdfs-project/hadoop-hdfs-client/src/main/java/org/apache/hadoop/hdfs/DFSInputStream.java","lineNumber":413,"sourceCode":"              \"Interrupted while getting the length.\");\n        }\n      }\n\n      // see if we ran out of retry time\n      if (sw.isRunning() && sw.now(TimeUnit.MILLISECONDS) > timeout) {\n        break;\n      }\n    }\n\n    // Namenode told us about these locations, but none know about the replica\n    // means that we hit the race between pipeline creation start and end.\n    // we require all 3 because some other exception could have happened\n    // on a DN that has it.  we want to report that error\n    if (replicaNotFoundCount == 0) {\n      return 0;\n    }\n\n    throw new CannotObtainBlockLengthException(locatedblock, src);\n  }\n\n  public long getFileLength() {\n    synchronized(infoLock) {\n      return locatedBlocks == null? 0:\n          locatedBlocks.getFileLength() + lastBlockBeingWrittenLength;\n    }\n  }\n\n  // Short circuit local reads are forbidden for files that are\n  // under construction.  See HDFS-2757.\n  boolean shortCircuitForbidden() {\n    synchronized(infoLock) {\n      return locatedBlocks.isUnderConstruction();\n    }\n  }\n\n  /**","sourceCodeStart":395,"sourceCodeEnd":431,"githubUrl":"https://github.com/apache/hadoop/blob/2add9630210752f88ceb1bb74eb65e37bf41da8e/hadoop-hdfs-project/hadoop-hdfs-client/src/main/java/org/apache/hadoop/hdfs/DFSInputStream.java#L395-L431","documentation":"CannotObtainBlockLengthException (an IOException subclass) thrown after the client queried the DataNodes listed for the last block: replicaNotFoundCount is non-zero, meaning at least one DN explicitly reported it does not hold the replica (ReplicaNotFoundException) and none could serve the length. The NameNode's metadata is ahead of (or orphaned from) what the DataNodes actually store.","triggerScenarios":"Reading a file under construction in the race window right after pipeline creation but before IBRs reach the NN; or all replicas of the block lost/deleted on the DNs while NN metadata still lists them.","commonSituations":"Tailing actively-written files on clusters with slow block reporting; reading right after DN restarts or disk failures that dropped replicas; post-recovery states where block metadata is stale.","solutions":["Retry the open/read after a short wait so pending block reports can land","Wait for the writer to close the file (last block finalized) before reading","Run hdfs fsck -files -blocks -locations on the file to see whether any replica actually exists","If all replicas are gone, recover via fsck (-move/-delete) or restore the file from its source/backup"],"exampleFix":null,"handlingStrategy":"retry","validationCode":null,"typeGuard":"static boolean isBlockLengthUnobtainable(IOException e) {\n  return e instanceof org.apache.hadoop.hdfs.CannotObtainBlockLengthException\n      || (e.getMessage() != null && e.getMessage().contains(\"Cannot obtain block length\"));\n}","tryCatchPattern":"for (int i = 0; i < 3; i++) {\n  try { in = fs.open(path); break; }\n  catch (IOException e) {\n    if (i == 2 || !isBlockLengthUnobtainable(e)) throw e;\n    Thread.sleep(3000L); // let block reports reach the NN\n  }\n}","preventionTips":["Read files under construction only after the writer closes them, when possible","After restarts, wait for block reports before reading freshly written files","Monitor under-replicated/missing blocks with fsck so stale metadata is caught early"],"tags":["hdfs","block-replica","under-construction","read"],"backgroundTag":"block-replica-missing","analyzedSha":"2add9630210752f88ceb1bb74eb65e37bf41da8e","analyzedAt":"2026-08-22T19:55:07.957Z","schemaVersion":2},"datasetVersion":"2026-08-23T01:17:44.959Z"}