apache/hadoop · error · IllegalArgumentException

Proxy error: fs.s3a.proxy.port set without fs.s3a.proxy.host

Error message

Proxy error: fs.s3a.proxy.port set without fs.s3a.proxy.host

What it means

IllegalArgumentException from AWSClientConfig's sync proxy setup when fs.s3a.proxy.port is configured (proxyPort >= 0) but fs.s3a.proxy.host is not set (the host branch was skipped, so the else-if on the port fires). A port without a host cannot describe a proxy endpoint, so client construction aborts before any S3 request is made.

Source

Thrown at hadoop-tools/hadoop-aws/src/main/java/org/apache/hadoop/fs/s3a/impl/AWSClientConfig.java:275

        String msg = "Proxy error: " + PROXY_USERNAME + " or " +
            PROXY_PASSWORD + " set without the other.";
        LOG.error(msg);
        throw new IllegalArgumentException(msg);
      }
      proxyConfigBuilder.username(proxyUsername);
      proxyConfigBuilder.password(proxyPassword);
      proxyConfigBuilder.ntlmDomain(conf.getTrimmed(PROXY_DOMAIN));
      proxyConfigBuilder.ntlmWorkstation(conf.getTrimmed(PROXY_WORKSTATION));
      if (LOG.isDebugEnabled()) {
        LOG.debug("Using proxy server {}:{} as user {} with password {} on "
                + "domain {} as workstation {}", proxyHost, proxyPort, proxyUsername, proxyPassword,
            PROXY_DOMAIN, PROXY_WORKSTATION);
      }
    } else if (proxyPort >= 0) {
      String msg =
          "Proxy error: " + PROXY_PORT + " set without " + PROXY_HOST;
      LOG.error(msg);
      throw new IllegalArgumentException(msg);
    }

    return proxyConfigBuilder.build();

  }

  /**
   * Configures the proxy for the async http client.
   * <p>
   * Although this is netty specific, it is part of the AWS SDK public API
   * and not any shaded netty internal class.
   * @param conf The Hadoop configuration
   * @param bucket Optional bucket to use to look up per-bucket proxy secrets
   * @return Proxy configuration
   * @throws IOException on any IO problem
   */
  public static software.amazon.awssdk.http.nio.netty.ProxyConfiguration
      createAsyncProxyConfiguration(Configuration conf,

View on GitHub (pinned to 2add963021)

Solutions

1. Set fs.s3a.proxy.host to the proxy hostname alongside the port
2. Or remove fs.s3a.proxy.port entirely if no proxy should be used
3. Check for typos and per-bucket overrides of the host key in core-site.xml

Example fix

<!-- before -->
<property><name>fs.s3a.proxy.port</name><value>8080</value></property>

<!-- after -->
<property><name>fs.s3a.proxy.host</name><value>proxy.corp</value></property>
<property><name>fs.s3a.proxy.port</name><value>8080</value></property>

String host = conf.getTrimmed("fs.s3a.proxy.host");
int port = conf.getInt("fs.s3a.proxy.port", -1);
if (port >= 0 && (host == null || host.isEmpty())) {
  throw new IllegalArgumentException("fs.s3a.proxy.port set without fs.s3a.proxy.host");
}

When it happens

Trigger: Initializing an S3A client on a node whose configuration sets fs.s3a.proxy.port (e.g. 8080) but leaves fs.s3a.proxy.host empty.

Common situations: Template config sets a shared port while the per-cluster host variable was never substituted; host key misspelled or placed under the wrong XML property; host removed when moving off a proxy but port left behind.

Related errors

• Proxy error: fs.s3a.proxy.username or fs.s3a.proxy.password
• Proxy error: incorrect fs.s3a.proxy.host or fs.s3a.proxy.por
• Unsupported block buffer "{name}"
• This committer does not work with the filesystem of type {sc
• Proxy error: fs.oss.proxy.username or fs.oss.proxy.password

AI-assisted analysis of apache/hadoop@2add963021 (2026-08-22). Data as JSON: /api/errors/6280842662fff413. Report an issue: GitHub.