apache/hadoop · critical · RuntimeException

ABFS endpoint is not set correctly : %s, Do not specify sche

Error message

ABFS endpoint is not set correctly : %s, Do not specify scheme when using {IP}:{PORT}

What it means

Thrown while building the ABFS service URI during filesystem initialization. When fs.azure.abfs.endpoint is set (used to target a raw IP endpoint such as a private endpoint, firewall appliance, or Azurite), the driver expects a bare {IP}:{PORT} value with no scheme: the string is split on ':' and must yield exactly two parts. Any value that splits into more (e.g. 'https://10.0.0.1:443' or an IPv6 literal with multiple colons) makes AzureBlobFileSystemStore throw this RuntimeException, so the FileSystem never comes up.

Source

Thrown at hadoop-tools/hadoop-azure/src/main/java/org/apache/hadoop/fs/azurebfs/AzureBlobFileSystemStore.java:493

    final URIBuilder uriBuilder = new URIBuilder();
    uriBuilder.setScheme(scheme);

    // For testing purposes, an IP address and port may be provided to override
    // the host specified in the FileSystem URI.  Also note that the format of
    // the Azure Storage Service URI changes from
    // http[s]://[account][domain-suffix]/[filesystem] to
    // http[s]://[ip]:[port]/[account]/[filesystem].
    String endPoint = abfsConfiguration.get(AZURE_ABFS_ENDPOINT);
    if (endPoint == null || !endPoint.contains(AbfsHttpConstants.COLON)) {
      uriBuilder.setHost(hostName);
      return uriBuilder;
    }

    // Split ip and port
    String[] data = endPoint.split(AbfsHttpConstants.COLON);
    if (data.length != 2) {
      throw new RuntimeException(String.format("ABFS endpoint is not set correctly : %s, "
              + "Do not specify scheme when using {IP}:{PORT}", endPoint));
    }
    uriBuilder.setHost(data[0].trim());
    uriBuilder.setPort(Integer.parseInt(data[1].trim()));
    uriBuilder.setPath("/" + UriUtils.extractAccountNameFromHostName(hostName));

    return uriBuilder;
  }

  public AbfsConfiguration getAbfsConfiguration() {
    return this.abfsConfiguration;
  }

  public Hashtable<String, String> getFilesystemProperties(
      TracingContext tracingContext) throws AzureBlobFileSystemException {
    try (AbfsPerfInfo perfInfo = startTracking("getFilesystemProperties",
            "getFilesystemProperties")) {
      LOG.debug("getFilesystemProperties for filesystem: {}",

View on GitHub (pinned to 2add963021)

Solutions

  1. Remove the scheme: set fs.azure.abfs.endpoint to exactly IP:PORT, e.g. '20.0.0.1:5000'. The scheme is chosen by fs.defaultFS (abfs:// = http, abfss:// = https), not by this key.
  2. For Azurite: fs.defaultFS=abfs://container@devstoreaccount1.dfs.core.windows.net, fs.azure.abfs.endpoint=127.0.0.1:10000, plus the well-known devstoreaccount1 key.
  3. Avoid IPv6 literals in this property; use a DNS hostname that resolves to the IP instead.
  4. Restart the JVM/service after editing core-site.xml so the store rebuilds the URI.

Example fix

<!-- before -->
<property><name>fs.azure.abfs.endpoint</name><value>https://20.0.0.1:5000</value></property>

<!-- after -->
<property><name>fs.azure.abfs.endpoint</name><value>20.0.0.1:5000</value></property>
Defensive patterns

Strategy: validation

Validate before calling

// validate before building the FileSystem
String ep = conf.get("fs.azure.abfs.endpoint");
if (ep != null && !ep.matches("^[^:/]+:[0-9]+$")) {
  throw new IllegalArgumentException(
      "fs.azure.abfs.endpoint must be bare IP:PORT (no scheme): " + ep);
}

Try / catch

try { FileSystem fs = path.getFileSystem(conf); } catch (RuntimeException e) { if (e.getMessage().contains("ABFS endpoint")) { /* fix fs.azure.abfs.endpoint to bare IP:PORT */ } throw e; }

Prevention

When it happens

Trigger: Setting fs.azure.abfs.endpoint to a URL with a scheme ('http://20.0.0.1:5000' splits into ['http','//20.0.0.1','5000'] = 3 parts), to an IPv6 address ('fe80::1:443' yields 4+ parts), or to any value with more than one colon. Note: a value with no colon at all does NOT throw - it is ignored and the account hostname is used.

Common situations: Pointing ABFS at Azurite or a storage-emulator IP and pasting the https:// URL from documentation into core-site.xml; migrating wasb-era configs that carried schemes; using IPv6 literals for private endpoints.

Related errors

AI-assisted analysis of apache/hadoop@2add963021 (2026-08-22). Data as JSON: /api/errors/9b2fc12521e3a539. Report an issue: GitHub.