apache/hadoop · error · CannotObtainBlockLengthException

Cannot obtain block length for {} of {}

Error message

Cannot obtain block length for {} of {}

What it means

CannotObtainBlockLengthException (an IOException subclass) thrown after the client queried the DataNodes listed for the last block: replicaNotFoundCount is non-zero, meaning at least one DN explicitly reported it does not hold the replica (ReplicaNotFoundException) and none could serve the length. The NameNode's metadata is ahead of (or orphaned from) what the DataNodes actually store.

Source

Thrown at hadoop-hdfs-project/hadoop-hdfs-client/src/main/java/org/apache/hadoop/hdfs/DFSInputStream.java:413

              "Interrupted while getting the length.");
        }
      }

      // see if we ran out of retry time
      if (sw.isRunning() && sw.now(TimeUnit.MILLISECONDS) > timeout) {
        break;
      }
    }

    // Namenode told us about these locations, but none know about the replica
    // means that we hit the race between pipeline creation start and end.
    // we require all 3 because some other exception could have happened
    // on a DN that has it.  we want to report that error
    if (replicaNotFoundCount == 0) {
      return 0;
    }

    throw new CannotObtainBlockLengthException(locatedblock, src);
  }

  public long getFileLength() {
    synchronized(infoLock) {
      return locatedBlocks == null? 0:
          locatedBlocks.getFileLength() + lastBlockBeingWrittenLength;
    }
  }

  // Short circuit local reads are forbidden for files that are
  // under construction.  See HDFS-2757.
  boolean shortCircuitForbidden() {
    synchronized(infoLock) {
      return locatedBlocks.isUnderConstruction();
    }
  }

  /**

View on GitHub (pinned to 2add963021)

Solutions

1. Retry the open/read after a short wait so pending block reports can land
2. Wait for the writer to close the file (last block finalized) before reading
3. Run hdfs fsck -files -blocks -locations on the file to see whether any replica actually exists
4. If all replicas are gone, recover via fsck (-move/-delete) or restore the file from its source/backup

static boolean isBlockLengthUnobtainable(IOException e) {
  return e instanceof org.apache.hadoop.hdfs.CannotObtainBlockLengthException
      || (e.getMessage() != null && e.getMessage().contains("Cannot obtain block length"));
}

for (int i = 0; i < 3; i++) {
  try { in = fs.open(path); break; }
  catch (IOException e) {
    if (i == 2 || !isBlockLengthUnobtainable(e)) throw e;
    Thread.sleep(3000L); // let block reports reach the NN
  }
}

When it happens

Trigger: Reading a file under construction in the race window right after pipeline creation but before IBRs reach the NN; or all replicas of the block lost/deleted on the DNs while NN metadata still lists them.

Common situations: Tailing actively-written files on clusters with slow block reporting; reading right after DN restarts or disk failures that dropped replicas; post-recovery states where block metadata is stale.

Related errors

• "concat: target file " + target + " is under construction"
• "concat: source file " + src + " is invalid or empty or unde
• Failed to {} {} for {} on {} because the file is under const
• Fail to get checksum, since file {} is under construction.
• Could not obtain the last block locations.

AI-assisted analysis of apache/hadoop@2add963021 (2026-08-22). Data as JSON: /api/errors/272c00343fdc0fea. Report an issue: GitHub.