apache/hadoop · error · IllegalArgumentException

URI: {} is not in the expected format

Error message

URI: {} is not in the expected format

What it means

StorageLocation.normalizeFileURI() converts a file-scheme URI through File.toURI(), strips a trailing slash, and re-parses it with new URI(...); a URISyntaxException there is wrapped in IllegalArgumentException('URI: ... is not in the expected format'). It is invoked from the StorageLocation constructor whenever the URI has no scheme or the 'file' scheme, i.e., for every local storage dir in dfs.datanode.data.dir.

Source

Thrown at hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/StorageLocation.java:92

  private StorageLocation(StorageType storageType, URI uri) {
    this.storageType = storageType;
    if (uri.getScheme() == null || uri.getScheme().equals("file")) {
      // make sure all URIs that point to a file have the same scheme
      uri = normalizeFileURI(uri);
    }
    baseURI = uri;
  }

  public static URI normalizeFileURI(URI uri) {
    try {
      File uriFile = new File(uri.getPath());
      String uriStr = uriFile.toURI().normalize().toString();
      if (uriStr.endsWith("/")) {
        uriStr = uriStr.substring(0, uriStr.length() - 1);
      }
      return new URI(uriStr);
    } catch (URISyntaxException e) {
      throw new IllegalArgumentException(
              "URI: " + uri + " is not in the expected format");
    }
  }

  public StorageType getStorageType() {
    return this.storageType;
  }

  public URI getUri() {
    return baseURI;
  }

  public URI getNormalizedUri() {
    return baseURI.normalize();
  }

  public boolean matchesStorageDirectory(StorageDirectory sd)
      throws IOException {

View on GitHub (pinned to 2add963021)

Solutions

1. Use plain absolute filesystem paths (or [DISK]/path) in dfs.datanode.data.dir - let Path/File do the encoding
2. If building URIs programmatically, construct them via new Path(location).toUri() instead of string concatenation
3. Sanitize the offending entry the message prints and restart the DataNode

Example fix

# before (hdfs-site.xml - bad encoding)
<name>dfs.datanode.data.dir</name><value>/data/1, /data/2#bad</value>

# after
<name>dfs.datanode.data.dir</name><value>/data/1,/data/2</value>

import java.io.File;
import java.net.URI;
URI safeNormalize(URI u) {
  try {
    String s = new File(u.getPath()).toURI().normalize().toString();
    if (s.endsWith("/")) s = s.substring(0, s.length() - 1);
    return new URI(s);
  } catch (java.net.URISyntaxException e) {
    throw new IllegalArgumentException("Unparseable storage URI: " + u, e);
  }
}
// pre-check config entries before passing them to StorageLocation.parse

try { StorageLocation.parse(entry); } catch (IllegalArgumentException e) { if (e.getMessage().contains("not in the expected format")) { /* reject config entry with clear operator message */ } else throw e; }

When it happens

Trigger: StorageLocation.parse()/new StorageLocation(type, uri) with a local path whose normalized file-URI string cannot be parsed by java.net.URI - in practice rare, since File.toURI() percent-encodes; reachable with unusual control characters or broken URI encodings produced by upstream code rather than plain paths.

Common situations: Malformed dfs.datanode.data.dir entries (unencoded special characters, stray quotes/backslashes from shell heredocs or config templating); programmatic construction of StorageLocation from a URI built with bad encoding; JVM differences in URI strictness after upgrades.

Related errors

• Capacity ratio{} is not between 0 to 1: {}
• Capacity ratio config is not with correct format: {}
• Cannot create directory {rootPath}
• Incompatible node types: storageType={storageType} but Stora
• Storage directory for location {} and block pool id {} does

AI-assisted analysis of apache/hadoop@2add963021 (2026-08-22). Data as JSON: /api/errors/96c37c9d539930cc. Report an issue: GitHub.