apache/hadoop · error · IOException

New block position {} must be greater than the last block po

Error message

New block position {} must be greater than the last block position {} for path {}

What it means

Error "New block position {} must be greater than the last block position {} for path {}" thrown in apache/hadoop.

Source

Thrown at hadoop-tools/hadoop-azure/src/main/java/org/apache/hadoop/fs/azurebfs/services/AzureBlobBlockManager.java:142

    }
    return committedBlockIdList;
  }

  /**
   * Adds a new block entry to the block entry list.
   * The block entry is added only if the position of the new block
   * is greater than the position of the last block in the list.
   *
   * @param blockId The ID of the new block to be added.
   * @param position The position of the new block in the stream.
   * @return The newly added {@link BlockEntry}.
   * @throws IOException If the position of the new block is not greater than the last block in the list.
   */
  private synchronized BlockEntry addNewEntry(String blockId, long position) throws IOException {
    if (!blockEntryList.isEmpty()) {
      BlockEntry lastEntry = blockEntryList.getLast();
      if (position <= lastEntry.getPosition()) {
        throw new IOException("New block position " + position  + " must be greater than the last block position "
            + lastEntry.getPosition() + " for path " + getAbfsOutputStream().getPath());
      }
    }
    BlockEntry blockEntry = new BlockEntry(blockId, position, AbfsBlockStatus.NEW);
    blockEntryList.addLast(blockEntry);
    LOG.debug("Added block {} at position {} with status NEW.", blockId, position);
    return blockEntry;
  }

  /**
   * Updates the status of an existing block entry to SUCCESS.
   * This method is used to mark a block as successfully processed.
   *
   * @param block The {@link AbfsBlock} whose status needs to be updated to SUCCESS.
   */
  protected synchronized void updateEntry(AbfsBlock block) {
    BlockEntry blockEntry = block.getBlockEntry();
    blockEntry.setStatus(AbfsBlockStatus.SUCCESS);

View on GitHub (pinned to 2add963021)

Solutions

1. Ensure blocks are committed in strictly increasing position order for the file.
2. Avoid concurrent writers to the same path, which interleave block positions.

When it happens

Trigger: A new block is added at a position not greater than the previous block, indicating concurrent or out-of-order writes.

Common situations: See trigger scenarios.

AI-assisted analysis of apache/hadoop@2add963021 (2026-08-22). Data as JSON: /api/errors/7b50c4f86131ca39. Report an issue: GitHub.