apache/hadoop · error · UnsupportedOperationException

Flush without blockIds not supported on Blob Endpoint

Error message

Flush without blockIds not supported on Blob Endpoint

What it means

AbfsBlobClient — the client bound to non-HNS (blob endpoint) accounts — cannot perform the DFS-style flush that commits uncommitted data by position. Committing data there is done exclusively via Put Block List, so the flush overload without blockIds unconditionally throws UnsupportedOperationException('Flush without blockIds not supported on Blob Endpoint'). The block-list flush (with the XML buffer of block ids) is the supported commit path.

Source

Thrown at hadoop-tools/hadoop-azure/src/main/java/org/apache/hadoop/fs/azurebfs/services/AbfsBlobClient.java:1057

   * @param isClose specify if this is the last flush to the file.
   * @param cachedSasToken to be used for the authenticating operation.
   * @param leaseId if there is an active lease on the path.
   * @param contextEncryptionAdapter to provide encryption context.
   * @param tracingContext for tracing the server calls.
   * @param blobMd5 the MD5 hash of the blob for integrity verification.
   * @return exception as this operation is not supported on Blob Endpoint.
   * @throws UnsupportedOperationException always.
   */
  @Override
  public AbfsRestOperation flush(final String path,
      final long position,
      final boolean retainUncommittedData,
      final boolean isClose,
      final String cachedSasToken,
      final String leaseId,
      final ContextEncryptionAdapter contextEncryptionAdapter,
      final TracingContext tracingContext, String blobMd5) throws AzureBlobFileSystemException {
    throw new UnsupportedOperationException(
        "Flush without blockIds not supported on Blob Endpoint");
  }

  /**
   * Get Rest Operation for API
   * <a href="../../../../site/markdown/blobEndpoint.md#put-block-list">Put Block List</a>.
   * The flush operation to commit the blocks.
   * @param buffer This has the xml in byte format with the blockIds to be flushed.
   * @param path The path to flush the data to.
   * @param isClose True when the stream is closed.
   * @param cachedSasToken The cachedSasToken if available.
   * @param leaseId The leaseId of the blob if available.
   * @param eTag The etag of the blob.
   * @param contextEncryptionAdapter to provide encryption context.
   * @param tracingContext for tracing the service call.
   * @param blobMd5 the MD5 hash of the blob for integrity verification.
   * @return executed rest operation containing response from server.
   * @throws AzureBlobFileSystemException if rest operation fails.

View on GitHub (pinned to 2add963021)

Solutions

1. On blob endpoints, commit via the block-list flush overload (flush with the blockIds buffer) — AzureBlobFileSystemOutputStream already does this internally
2. Branch your code on client.getIsNamespaceEnabled(): use the DFS flush only for HNS accounts
3. Prefer the public FileSystem/FSDataOutputStream API (hflush/hsync) instead of AbfsClient internals

Example fix

// before: DFS-style flush on a blob endpoint -> UnsupportedOperationException
abfsClient.flush(path, position, false, true, null, null, null, tracingContext);

// after: route by account capability
if (abfsClient.getIsNamespaceEnabled()) {
  abfsClient.flush(path, position, false, true, null, null, null, tracingContext);
} else {
  // commit blocks with the block-list flush used by the stream layer
  out.hsync(); // or AbfsBlobClient flush(buffer, path, ...) with blockIds
}

if (!abfsClient.getIsNamespaceEnabled()) {
  // blob endpoint: commit via block-list flush (hflush/hsync on the stream), never the DFS flush
  out.hsync();
} else {
  abfsClient.flush(path, position, false, true, null, null, null, tracingContext);
}

try {
  abfsClient.flush(path, position, false, true, null, null, null, tracingContext);
} catch (UnsupportedOperationException e) {
  // non-HNS account: fall back to the block-list commit path
}

When it happens

Trigger: Directly invoking AbfsClient/AbfsBlobClient.flush(path, position, retainUncommittedData, isClose, cachedSasToken, leaseId, ...) — the no-blockIds overload — against a non-HNS account. Normal stream writers branch to the block-list variant; this fires only from custom or ported code calling the internal API directly.

Common situations: Custom code or forks calling AbfsClient internals; code written against the DFS flush signature reused on a blob-endpoint account; version upgrades that reshaped flush overloads.

Related errors

• Cannot create file {} because parent folder does not exist.
• PathConflict
• Parallel access to the create path detected. Failing request
• Parallel access to the create path detected. Failing request
• FNS-Blob rename was not successful for source and destinatio

AI-assisted analysis of apache/hadoop@2add963021 (2026-08-22). Data as JSON: /api/errors/09382617da2ca50b. Report an issue: GitHub.