apache/hadoop · error · IllegalArgumentException

Proxy error: incorrect fs.s3a.proxy.host or fs.s3a.proxy.por

Error message

Proxy error: incorrect fs.s3a.proxy.host or fs.s3a.proxy.port

What it means

IllegalArgumentException from AWSClientConfig.buildURI when java.net.URI rejects the proxy endpoint assembled from scheme, host and port (URISyntaxException). buildURI is used to construct the proxy endpoint URI for both sync and async clients (for example when the host is set without a port it defaults to http/80 with a logged warning). The message deliberately names fs.s3a.proxy.host and fs.s3a.proxy.port as the suspects.

Source

Thrown at hadoop-tools/hadoop-aws/src/main/java/org/apache/hadoop/fs/s3a/impl/AWSClientConfig.java:367

    return proxyConfigBuilder.build();
  }

  /**
   * Builds a URI, throws an IllegalArgumentException in case of errors.
   *
   * @param host proxy host
   * @param port proxy port
   * @return uri with host and port
   */
  private static URI buildURI(String scheme, String host, int port) {
    try {
      return new URI(scheme, null, host, port, null, null, null);
    } catch (URISyntaxException e) {
      String msg =
          "Proxy error: incorrect " + PROXY_HOST + " or " + PROXY_PORT;
      LOG.error(msg);
      throw new IllegalArgumentException(msg);
    }
  }

  /**
   * Initializes the User-Agent header to send in HTTP requests to AWS
   * services.  We always include the Hadoop version number.  The user also
   * may set an optional custom prefix to put in front of the Hadoop version
   * number.  The AWS SDK internally appends its own information, which seems
   * to include the AWS SDK version, OS and JVM version.
   *
   * @param conf Hadoop configuration
   * @param clientConfig AWS SDK configuration to update
   */
  private static void initUserAgent(Configuration conf,
      ClientOverrideConfiguration.Builder clientConfig) {
    String userAgent = "Hadoop " + VersionInfo.getVersion();
    String userAgentPrefix = conf.getTrimmed(USER_AGENT_PREFIX, "");
    if (!userAgentPrefix.isEmpty()) {

View on GitHub (pinned to 2add963021)

Solutions

1. Set fs.s3a.proxy.host to a bare hostname or IP (no scheme, no path, no port)
2. Set fs.s3a.proxy.port to a plain integer in 0-65535
3. Trim the values / fix the templating that injected whitespace or the full URL

Example fix

<!-- before -->
<property><name>fs.s3a.proxy.host</name><value>http://proxy.corp:8080</value></property>

<!-- after -->
<property><name>fs.s3a.proxy.host</name><value>proxy.corp</value></property>
<property><name>fs.s3a.proxy.port</name><value>8080</value></property>

String host = conf.getTrimmed("fs.s3a.proxy.host");
int port = conf.getInt("fs.s3a.proxy.port", -1);
try {
  new java.net.URI("http", null, host, port, null, null, null);
} catch (java.net.URISyntaxException bad) {
  throw new IllegalArgumentException("Proxy host/port not URI-safe: " + host + ":" + port, bad);
}

try {
  FileSystem fs = path.getFileSystem(conf);
} catch (IllegalArgumentException e) {
  if (e.getMessage() != null && e.getMessage().contains("incorrect fs.s3a.proxy")) {
    // host or port contains illegal characters/range: fix values, do not retry
    LOG.error("Proxy host/port rejected by URI parser", e);
  }
}

When it happens

Trigger: fs.s3a.proxy.host contains characters illegal in a URI host -- spaces, 'http://' prefixes, slashes, underscores in some strict parses -- or the port is outside 0-65535, so 'new URI(scheme, null, host, port, ...)' throws.

Common situations: Host pasted as 'http://proxy.corp:8080' instead of the bare hostname; whitespace or a trailing slash in the XML value; port parsed from a templated variable that produced a negative or non-numeric value.

Related errors

• Proxy error: fs.s3a.proxy.username or fs.s3a.proxy.password
• Proxy error: fs.s3a.proxy.port set without fs.s3a.proxy.host
• Unsupported block buffer "{name}"
• Does not contain a valid host:port authority: ${target} (con
• This committer does not work with the filesystem of type {sc

AI-assisted analysis of apache/hadoop@2add963021 (2026-08-22). Data as JSON: /api/errors/abe27be0654eb852. Report an issue: GitHub.