apache/hadoop · error · IllegalArgumentException

DFS client does not support blockId

Error message

DFS client does not support blockId

What it means

AbfsBlock.getBlockId() unconditionally throws IllegalArgumentException. Block IDs exist only for blob-endpoint 'stage block' uploads (AbfsBlobBlock); a plain AbfsBlock belongs to the DFS (abfs://) client, which streams via append and has no block-id concept. Calling this method is a type misuse, not a runtime storage failure.

Source

Thrown at hadoop-tools/hadoop-azure/src/main/java/org/apache/hadoop/fs/azurebfs/services/AbfsBlock.java:122

   * @return the offset of the block.
   */
  public Long getOffset() {
    return offset;
  }

  @Override
  public void close() throws IOException {
    if (activeBlock != null) {
      activeBlock.close();
    }
  }

  /**
   * Returns blockId for the block.
   * @return blockId.
   */
  public String getBlockId() {
    throw new IllegalArgumentException("DFS client does not support blockId");
  }

  /**
   * Gets the AbfsOutputStream.
   *
   * @return the AbfsOutputStream.
   */
  public AbfsOutputStream getOutputStream() {
    return outputStream;
  }

  /**
   * Sets the AbfsOutputStream.
   *
   * @param outputStream the AbfsOutputStream to set.
   */
  public void setOutputStream(final AbfsOutputStream outputStream) {
    this.outputStream = outputStream;

View on GitHub (pinned to 2add963021)

Solutions

1. Branch on the concrete block/client type: only AbfsBlobBlock (blob endpoint) exposes a valid blockId.
2. Prefer instanceof checks over blind casts to AbfsBlobBlock before any block-id API.
3. Confirm which endpoint the filesystem uses (abfs:// dfs vs blob endpoint) and route block-id logic accordingly.

Example fix

// before
for (AbfsBlock b : blocks) { commitBlockId(b.getBlockId()); } // DFS blocks throw
// after
for (AbfsBlock b : blocks) {
  if (b instanceof AbfsBlobBlock) {
    commitBlockId(((AbfsBlobBlock) b).getBlockId());
  }
}

if (block instanceof AbfsBlobBlock) {
  String id = ((AbfsBlobBlock) block).getBlockId();
} else {
  // DFS-endpoint block: no blockId concept, use append path
}

try { useBlockId(block.getBlockId()); }
catch (IllegalArgumentException ex) {
  if (ex.getMessage().contains("does not support blockId")) { /* wrong endpoint type */ }
  else throw ex;
}

When it happens

Trigger: Code that downcasts or handles AbfsBlock generically and calls getBlockId() — e.g. committing staged blocks — while the stream was created against the dfs (abfs://) endpoint rather than the blob endpoint.

Common situations: Custom output committers or instrumentation walking AbfsOutputStream blocks; refactors that assume blob-endpoint semantics on HNS accounts using the dfs endpoint; code paths shared between AbfsClient and AbfsBlobClient.

Related errors

• Flush without blockIds not supported on Blob Endpoint
• SetOwner operation is only supported on HNS enabled Accounts
• SetPermission operation is only supported on HNS enabled Acc
• SetAcl operation is only supported on HNS enabled Accounts.
• GetAclStatus operation is only supported on HNS enabled Acco

AI-assisted analysis of apache/hadoop@2add963021 (2026-08-22). Data as JSON: /api/errors/43dc0d405f34ce58. Report an issue: GitHub.