apache/hadoop · error · InvalidConfigurationValueException

The ingress service type must be configured as DFS

Error message

The ingress service type must be configured as DFS

What it means

AbfsOutputStream.createNewHandler throws InvalidConfigurationValueException("The ingress service type must be configured as DFS") when three conditions coincide: fs.azure.enable.dfstoblob.fallback is true, the account is hierarchical-namespace enabled (HNS), and the configured ingress service type (fs.azure.ingress.service.type) at stream creation is not DFS. The DFS-to-blob fallback uploads via DFS and falls back to blob endpoints, so a separate blob ingress on an HNS account is rejected up front.

Source

Thrown at hadoop-tools/hadoop-azure/src/main/java/org/apache/hadoop/fs/azurebfs/services/AbfsOutputStream.java:302

      }
    }
    // If ingressHandler is null, no lock is needed; safely initialize it outside the lock
    return createNewHandler(serviceType, blockFactory, bufferSize, isSwitch, blockManager);
  }

  // Helper method to create a new handler, used in both scenarios (locked and unlocked)
  private AzureIngressHandler createNewHandler(AbfsServiceType serviceType,
      DataBlocks.BlockFactory blockFactory,
      int bufferSize,
      boolean isSwitch,
      AzureBlockManager blockManager) throws IOException {
    this.client = clientHandler.getClient(serviceType);

    // Check ingress service type is also set to DFS along with enabling the config for fallback
    // Separate ingress service type is only allowed for HNS accounts
    if (isDFSToBlobFallbackEnabled && client.getIsNamespaceEnabled()
        && serviceTypeAtInit != AbfsServiceType.DFS) {
      throw new InvalidConfigurationValueException(
          "The ingress service type must be configured as DFS");
    }
    if (isDFSToBlobFallbackEnabled && !isSwitch) {
      ingressHandler = new AzureDfsToBlobIngressFallbackHandler(this,
          blockFactory, bufferSize, eTag, clientHandler);
    } else if (serviceType == AbfsServiceType.BLOB) {
      ingressHandler = new AzureBlobIngressHandler(this, blockFactory,
          bufferSize, eTag, clientHandler, blockManager);
    } else {
      ingressHandler = new AzureDFSIngressHandler(this, blockFactory,
          bufferSize, eTag, clientHandler);
    }
    if (isSwitch) {
      switchCompleted = true;
    }
    return ingressHandler;
  }

View on GitHub (pinned to 2add963021)

Solutions

1. Set fs.azure.ingress.service.type=DFS (or remove it) when fs.azure.enable.dfstoblob.fallback is true
2. Or set fs.azure.enable.dfstoblob.fallback=false if blob ingress is what you want
3. Remember the guard applies to HNS accounts only — a flat (non-HNS) account does not trip it

Example fix

<!-- before: conflicting combination on an HNS account -->
<property>
  <name>fs.azure.enable.dfstoblob.fallback</name>
  <value>true</value>
</property>
<property>
  <name>fs.azure.ingress.service.type</name>
  <value>Blob</value>
</property>

<!-- after -->
<property>
  <name>fs.azure.enable.dfstoblob.fallback</name>
  <value>true</value>
</property>
<property>
  <name>fs.azure.ingress.service.type</name>
  <value>DFS</value>
</property>

// Reject the conflicting combination before mounting the FS
boolean fallback = conf.getBoolean("fs.azure.enable.dfstoblob.fallback", false);
String ingress = conf.get("fs.azure.ingress.service.type");
if (fallback && hnsEnabled && !"DFS".equalsIgnoreCase(ingress == null ? "DFS" : ingress)) {
  throw new IllegalArgumentException(
      "fs.azure.ingress.service.type must be DFS when"
      + " fs.azure.enable.dfstoblob.fallback=true on an HNS account");
}

When it happens

Trigger: Setting fs.azure.ingress.service.type=BLOB (to write through the blob endpoint) while also enabling fs.azure.enable.dfstoblob.fallback=true on an HNS-enabled storage account; enabling the fallback knob in tuning guides without checking the ingress type already set for throughput reasons.

Common situations: Performance tuning where blob ingress was configured first and the fallback flag added later; config reuse across accounts where one account is HNS and another is not; version upgrades that added the guard to previously permissive combinations.

Related errors

• Lease desired but no lease threads configured, set fs.azure.
• - account name is not fully qualified.
• Exception while initializing metric credentials
• MD5 algorithm not available
• Failed to determine account type

AI-assisted analysis of apache/hadoop@2add963021 (2026-08-22). Data as JSON: /api/errors/c24b959e7e9e0cf6. Report an issue: GitHub.