apache/hadoop · error · IllegalArgumentException

Invalid scheme [{0}] it should be 'webhdfs' or 'swebhdfs'

Error message

Invalid scheme [{0}] it should be 'webhdfs' or 'swebhdfs'

What it means

HttpFSUtils.createURL() maps the FileSystem URI scheme to a transport: webhdfs becomes http, swebhdfs becomes https. Any other scheme throws IllegalArgumentException before a network connection is made. Note the message formats the whole URI into {0} (the code passes `uri`, not `uri.getScheme()`), so the bracketed value you see is the full URI; a URI with no scheme at all NPEs earlier at uri.getScheme().equalsIgnoreCase(...).

Source

Thrown at hadoop-hdfs-project/hadoop-hdfs-httpfs/src/main/java/org/apache/hadoop/fs/http/client/HttpFSUtils.java:92

   * @param path the file path.
   * @param params the query string parameters.
   * @param multiValuedParams multi valued parameters of the query string
   *
   * @return URL a <code>URL</code> for the HttpFSServer server,
   *
   * @throws IOException thrown if an IO error occurs.
   */
  static URL createURL(Path path, Map<String, String> params, Map<String, 
      List<String>> multiValuedParams) throws IOException {
    URI uri = path.toUri();
    String realScheme;
    if (uri.getScheme().equalsIgnoreCase(HttpFSFileSystem.SCHEME)) {
      realScheme = "http";
    } else if (uri.getScheme().equalsIgnoreCase(HttpsFSFileSystem.SCHEME)) {
      realScheme = "https";

    } else {
      throw new IllegalArgumentException(MessageFormat.format(
        "Invalid scheme [{0}] it should be '" + HttpFSFileSystem.SCHEME + "' " +
            "or '" + HttpsFSFileSystem.SCHEME + "'", uri));
    }
    StringBuilder sb = new StringBuilder();
    sb.append(realScheme).append("://").append(uri.getAuthority()).
      append(SERVICE_PATH).append(uri.getPath());

    String separator = "?";
    for (Map.Entry<String, String> entry : params.entrySet()) {
      sb.append(separator).append(entry.getKey()).append("=").
        append(URLEncoder.encode(entry.getValue(), "UTF8"));
      separator = "&";
    }
    if (multiValuedParams != null) {
      for (Map.Entry<String, List<String>> multiValuedEntry : 
        multiValuedParams.entrySet()) {
        String name = URLEncoder.encode(multiValuedEntry.getKey(), "UTF-8");
        List<String> values = multiValuedEntry.getValue();

View on GitHub (pinned to 2add963021)

Solutions

1. Use webhdfs://host:port for plain HTTP or swebhdfs://host:port for TLS in the FileSystem URI
2. Check the code path that builds the URI (fs.defaultFS, hard-coded URI, Path) and correct the scheme string
3. If you see an NPE instead, the URI has no scheme at all: qualify it or set fs.defaultFS so FileSystem.get resolves one
4. Verify the port is the HttpFS port (default 14000) / NameNode webhdfs port (9870) for the chosen scheme

Example fix

// before
FileSystem fs = FileSystem.get(new URI("hdfs://httpfs-host:14000"), conf);
// after
FileSystem fs = FileSystem.get(new URI("webhdfs://httpfs-host:14000"), conf);

URI u = new URI(fsUrl);
String scheme = u.getScheme();
if (scheme == null || !(scheme.equalsIgnoreCase("webhdfs") || scheme.equalsIgnoreCase("swebhdfs"))) {
  throw new IllegalArgumentException("HttpFS client requires webhdfs:// or swebhdfs://, got: " + scheme);
}
FileSystem fs = FileSystem.get(u, conf);

static boolean isWebhdfsUri(URI u) {
  String s = u.getScheme();
  return "webhdfs".equalsIgnoreCase(s) || "swebhdfs".equalsIgnoreCase(s);
}

try {
  FileSystem fs = FileSystem.get(new URI(fsUrl), conf);
} catch (IllegalArgumentException e) {
  if (e.getMessage() != null && e.getMessage().contains("Invalid scheme")) {
    throw new ConfigurationException("Set fsUrl to webhdfs://host:14000 or swebhdfs://host:port, was: " + fsUrl, e);
  }
  throw e;
}

When it happens

Trigger: FileSystem.get(new URI("hdfs://host:14000", ...), conf) or any operation routed through HttpFSUtils.createURL (getFileStatus, open, listStatus, create...) when the fs URI scheme is hdfs://, file://, httpfs://, or a typo like webhdf://.

Common situations: Copying the hdfs:// fs.defaultFS value from core-site.xml into code that instantiates the WebHDFS/HttpFS client; assuming the HttpFS proxy accepts its own 'httpfs' scheme; missing scheme because Path was built from a bare string after config changes.

Related errors

• Invalid DFS directory name {0}
• Bad rule definition: {bad_lines}
• Failed to parse "{str}" for the parameter {varName}. The va
• Invalid value: "{str}" does not belong to the domain {domain
• Missing HTTP 'Location' header for [{0}]

AI-assisted analysis of apache/hadoop@2add963021 (2026-08-22). Data as JSON: /api/errors/9fbb7b23630448c7. Report an issue: GitHub.