apache/hadoop · error · UnsupportedOperationException

Flush with blockIds not supported on DFS Endpoint

Error message

Flush with blockIds not supported on DFS Endpoint

What it means

Error "Flush with blockIds not supported on DFS Endpoint" thrown in apache/hadoop.

Source

Thrown at hadoop-tools/hadoop-azure/src/main/java/org/apache/hadoop/fs/azurebfs/services/AbfsDfsClient.java:921

   * @param isClose whether this is the last flush operation to the file.
   * @param cachedSasToken the SAS token to authenticate the operation.
   * @param leaseId the lease ID, if an active lease exists on the path.
   * @param eTag the ETag for concurrency control to ensure the flush is applied to the correct file version.
   * @param contextEncryptionAdapter the adapter providing the encryption context.
   * @param tracingContext the tracing context for tracking server calls.
   * @param blobMd5 the MD5 hash of the blob for integrity verification.
   * @throws UnsupportedOperationException if flush with blockIds is called on a DFS endpoint.
   */
  @Override
  public AbfsRestOperation flush(byte[] buffer,
      final String path,
      boolean isClose,
      final String cachedSasToken,
      final String leaseId,
      final String eTag,
      final ContextEncryptionAdapter contextEncryptionAdapter,
      final TracingContext tracingContext, String blobMd5) throws AzureBlobFileSystemException {
    throw new UnsupportedOperationException(
        "Flush with blockIds not supported on DFS Endpoint");
  }

  /**
   * Get Rest Operation for API
   * <a href="https://learn.microsoft.com/en-us/rest/api/storageservices/datalakestoragegen2/path/update">
   *   Path - Update</a>.
   * Set the properties of a file or directory.
   * @param path on which properties have to be set.
   * @param properties list of metadata key-value pairs.
   * @param tracingContext for tracing the server calls.
   * @param contextEncryptionAdapter to provide encryption context.
   * @return executed rest operation containing response from server.
   * @throws AzureBlobFileSystemException if rest operation fails.
   */
  @Override
  public AbfsRestOperation setPathProperties(final String path,
      final Hashtable<String, String> properties,

View on GitHub (pinned to 2add963021)

Solutions

1. Do not pass block ids to flush() when using the DFS endpoint; use plain flush/hflush.
2. If block-id flush semantics are required, write through the Blob endpoint client instead.

When it happens

Trigger: Application code calls a flush variant carrying block ids against an ABFS DFS-endpoint stream.

Common situations: See trigger scenarios.

AI-assisted analysis of apache/hadoop@2add963021 (2026-08-22). Data as JSON: /api/errors/55b47d29bf2bf8d6. Report an issue: GitHub.