apache/hadoop · error · IOException

Blocklist for {} has changed!

Error message

Blocklist for {} has changed!

What it means

During an openInfo(true) refresh, the client walks the cached LocatedBlocks and the newly fetched ones in parallel and requires identical block IDs in order. Any mismatch throws this IOException: the inode at src now points at different blocks, i.e. the file was replaced (deleted and recreated, truncated to zero and rewritten) while this stream held the old block list.

Source

Thrown at hadoop-hdfs-project/hadoop-hdfs-client/src/main/java/org/apache/hadoop/hdfs/DFSInputStream.java:301

    }
  }

  private LocatedBlocks fetchAndCheckLocatedBlocks(LocatedBlocks existing)
      throws IOException {
    LocatedBlocks newInfo = dfsClient.getLocatedBlocks(src, 0);

    DFSClient.LOG.debug("newInfo = {}", newInfo);
    if (newInfo == null) {
      throw new IOException("Cannot open filename " + src);
    }

    if (existing != null) {
      Iterator<LocatedBlock> oldIter =
          existing.getLocatedBlocks().iterator();
      Iterator<LocatedBlock> newIter = newInfo.getLocatedBlocks().iterator();
      while (oldIter.hasNext() && newIter.hasNext()) {
        if (!oldIter.next().getBlock().equals(newIter.next().getBlock())) {
          throw new IOException("Blocklist for " + src + " has changed!");
        }
      }
    }

    return newInfo;
  }

  private long getLastBlockLength(LocatedBlocks blocks) throws IOException{
    long lastBlockBeingWrittenLength = 0;
    if (!blocks.isLastBlockComplete()) {
      final LocatedBlock last = blocks.getLastLocatedBlock();
      if (last != null) {
        if (last.getLocations().length == 0) {
          if (last.getBlockSize() == 0) {
            // if the length is zero, then no data has been written to
            // datanode. So no need to wait for the locations.
            return 0;
          }

View on GitHub (pinned to 2add963021)

Solutions

1. Reopen the stream to pick up the new inode/block list, then re-seek
2. Have writers publish new versions with atomic rename instead of delete+recreate so readers keep a stable inode
3. Use append to grow files rather than recreating them under the same path

static boolean isBlocklistChanged(IOException e) {
  return e.getMessage() != null && e.getMessage().contains("has changed");
}

try {
  /* read from in */
} catch (IOException e) {
  if (isBlocklistChanged(e)) {
    in.close();
    in = fs.open(path);   // pick up the new inode/block list
    in.seek(pos);
  } else throw e;
}

When it happens

Trigger: A read-path block refresh on a file that another client concurrently recreated under the same name; overwrite-style writers (write temp, delete target, rename) racing a reader that already opened the old inode.

Common situations: Reading job output while a rerun job overwrites it (mapreduce output overwrite); log rotation implemented as delete+recreate; concurrent truncate followed by rewrite.

Related errors

• Another {name} is running.
• Commit or complete block {commitBlock}, whereas it is under
• Interrupted while running disk check
• PLAN_ALREADY_IN_PROGRESS
• rur.getRecoveryID() >= recoveryId = {recoveryId}, block={blo

AI-assisted analysis of apache/hadoop@2add963021 (2026-08-22). Data as JSON: /api/errors/fe45df508d80b050. Report an issue: GitHub.