apache/hadoop · error · IllegalArgumentException

Received invalid http response: {statusCode}, text = {text}

Error message

Received invalid http response: {statusCode}, text = {text}

What it means

CredentialBasedAccessTokenProvider.refresh posts the client credential to dfs.webhdfs.oauth2.refresh.url to obtain an access token; any HTTP status other than 200 throws IllegalArgumentException embedding the status code and full response text. This is the client-credentials grant variant of the same check as ConfRefreshTokenBasedAccessTokenProvider (message and flow identical, provider differs). The embedded body carries the IdP's exact error (invalid_client, invalid_request, unauthorized_client, 5xx...).

Source

Thrown at hadoop-hdfs-project/hadoop-hdfs-client/src/main/java/org/apache/hadoop/hdfs/web/oauth2/CredentialBasedAccessTokenProvider.java:124

  void refresh() throws IOException {
    final List<NameValuePair> pairs = new ArrayList<>();
    pairs.add(new BasicNameValuePair(CLIENT_SECRET, getCredential()));
    pairs.add(new BasicNameValuePair(GRANT_TYPE, CLIENT_CREDENTIALS));
    pairs.add(new BasicNameValuePair(CLIENT_ID, clientId));
    final RequestConfig config = RequestConfig.custom()
        .setConnectTimeout(URLConnectionFactory.DEFAULT_SOCKET_TIMEOUT)
        .setConnectionRequestTimeout(URLConnectionFactory.DEFAULT_SOCKET_TIMEOUT)
        .setSocketTimeout(URLConnectionFactory.DEFAULT_SOCKET_TIMEOUT)
        .build();
    try (CloseableHttpClient client =
             HttpClientBuilder.create().setDefaultRequestConfig(config).build()) {
      final HttpPost httpPost = new HttpPost(refreshURL);
      httpPost.setEntity(new UrlEncodedFormEntity(pairs, StandardCharsets.UTF_8));
      httpPost.setHeader(HttpHeaders.CONTENT_TYPE, URLENCODED);
      try (CloseableHttpResponse response = client.execute(httpPost)) {
        final int statusCode = response.getStatusLine().getStatusCode();
        if (statusCode != HttpStatus.SC_OK) {
          throw new IllegalArgumentException(
              "Received invalid http response: " + statusCode + ", text = " +
                  EntityUtils.toString(response.getEntity()));
        }
        Map<?, ?> responseBody = JsonSerialization.mapReader().readValue(
            EntityUtils.toString(response.getEntity()));

        String newExpiresIn = responseBody.get(EXPIRES_IN).toString();
        timer.setExpiresIn(newExpiresIn);

        accessToken = responseBody.get(ACCESS_TOKEN).toString();
      }
    } catch (RuntimeException e) {
      throw new IOException("Unable to obtain access token from credential", e);
    } catch (Exception e) {
      throw new IOException("Unable to obtain access token from credential", e);
    }
  }
}

View on GitHub (pinned to 2add963021)

Solutions

1. Read the response text inside the exception message and act on the OAuth error code: invalid_client -> update dfs.webhdfs.oauth2.credential / client id; unauthorized_client -> grant the flow to the app
2. Confirm dfs.webhdfs.oauth2.refresh.url and dfs.webhdfs.oauth2.client.id match the IdP app registration
3. Validate the credential independently with curl -d 'grant_type=client_credentials&client_id=...&client_secret=...' <url>
4. Rotate and redistribute the credential if it has expired or been revoked

Example fix

<!-- before -->
<property><name>dfs.webhdfs.oauth2.credential</name><value>expired-secret</value></property>
<!-- after: current secret from the IdP app registration -->
<property><name>dfs.webhdfs.oauth2.credential</name><value>current-secret</value></property>

assertNotNull(conf.get("dfs.webhdfs.oauth2.credential"), "credential");
assertNotNull(conf.get("dfs.webhdfs.oauth2.client.id"), "client id");
assertNotNull(conf.get("dfs.webhdfs.oauth2.refresh.url"), "token url");

try {
  fs.open(p);
} catch (IOException e) {
  Throwable root = e.getCause() != null ? e.getCause() : e;
  if (root instanceof IllegalArgumentException
      && root.getMessage().contains("Received invalid http response")) {
    // status + body are in the message: invalid_client -> rotate credential,
    // 5xx -> IdP outage, retry later
  }
}

When it happens

Trigger: refresh() running on token expiry when the token endpoint replies non-200: wrong or expired dfs.webhdfs.oauth2.credential secret, wrong dfs.webhdfs.oauth2.client.id, endpoint URL typo, or IdP outage (503).

Common situations: Credential rotated in the IdP but dfs.webhdfs.oauth2.credential in core-site.xml not updated; app not granted the OAuth2 client-credentials flow (unauthorized_client); environment-specific URLs mixed up; AAD/Google endpoint format changes.

Related errors

• Received invalid http response: {statusCode}, text = {text}
• Unable to obtain access token from credential
• Unable to load OAuth2 connection factory.
• Credential has not been provided in configuration
• Exception while refreshing access token

AI-assisted analysis of apache/hadoop@2add963021 (2026-08-22). Data as JSON: /api/errors/f80f875dd56b7eaf. Report an issue: GitHub.