apache/hadoop · error · IllegalArgumentException

Received invalid http response: {statusCode}, text = {text}

Error message

Received invalid http response: {statusCode}, text = {text}

What it means

ConfRefreshTokenBasedAccessTokenProvider.refresh posts a refresh_token grant to dfs.webhdfs.oauth2.refresh.url; any HTTP status other than 200 throws IllegalArgumentException embedding the status code and the raw response body. The body text is the key diagnostic — OAuth2 errors like invalid_grant (expired/revoked refresh token), invalid_client (bad dfs.webhdfs.oauth2.client.id), or 5xx from the identity provider appear there verbatim.

Source

Thrown at hadoop-hdfs-project/hadoop-hdfs-client/src/main/java/org/apache/hadoop/hdfs/web/oauth2/ConfRefreshTokenBasedAccessTokenProvider.java:130

  void refresh() throws IOException {
    final List<NameValuePair> pairs = new ArrayList<>();
    pairs.add(new BasicNameValuePair(GRANT_TYPE, REFRESH_TOKEN));
    pairs.add(new BasicNameValuePair(REFRESH_TOKEN, refreshToken));
    pairs.add(new BasicNameValuePair(CLIENT_ID, clientId));
    final RequestConfig config = RequestConfig.custom()
        .setConnectTimeout(URLConnectionFactory.DEFAULT_SOCKET_TIMEOUT)
        .setConnectionRequestTimeout(URLConnectionFactory.DEFAULT_SOCKET_TIMEOUT)
        .setSocketTimeout(URLConnectionFactory.DEFAULT_SOCKET_TIMEOUT)
        .build();
    try (CloseableHttpClient client =
             HttpClientBuilder.create().setDefaultRequestConfig(config).build()) {
      final HttpPost httpPost = new HttpPost(refreshURL);
      httpPost.setEntity(new UrlEncodedFormEntity(pairs, StandardCharsets.UTF_8));
      httpPost.setHeader(HttpHeaders.CONTENT_TYPE, URLENCODED);
      try (CloseableHttpResponse response = client.execute(httpPost)) {
        final int statusCode = response.getStatusLine().getStatusCode();
        if (statusCode != HttpStatus.SC_OK) {
          throw new IllegalArgumentException(
              "Received invalid http response: " + statusCode + ", text = " +
                  EntityUtils.toString(response.getEntity()));
        }
        Map<?, ?> responseBody = JsonSerialization.mapReader().readValue(
            EntityUtils.toString(response.getEntity()));

        String newExpiresIn = responseBody.get(EXPIRES_IN).toString();
        accessTokenTimer.setExpiresIn(newExpiresIn);

        accessToken = responseBody.get(ACCESS_TOKEN).toString();
      }
    } catch (RuntimeException e) {
      throw new IOException("Exception while refreshing access token", e);
    } catch (Exception e) {
      throw new IOException("Exception while refreshing access token", e);
    }
  }

View on GitHub (pinned to 2add963021)

Solutions

1. Read the embedded response text in the exception message: invalid_grant means obtain and configure a fresh refresh token in dfs.webhdfs.oauth2.refresh.token
2. Verify dfs.webhdfs.oauth2.refresh.url points at the real token endpoint (e.g. https://login.microsoftonline.com/<tenant>/oauth2/token) and dfs.webhdfs.oauth2.client.id matches the registered application
3. Reproduce outside Hadoop: curl -d 'grant_type=refresh_token&refresh_token=...' ... to see the exact IdP error
4. If the body is HTML, you are hitting a login/redirect page — fix the URL; if 5xx, the IdP is down, retry later

Example fix

<!-- before: stale refresh token in core-site.xml -->
<property><name>dfs.webhdfs.oauth2.refresh.token</name><value>old-token</value></property>
<!-- after: replace with the token freshly issued by the IdP -->
<property><name>dfs.webhdfs.oauth2.refresh.token</name><value>AQEAA...</value></property>

// No safe pre-check of the IdP exists; validate local config instead
assertNotNull(conf.get("dfs.webhdfs.oauth2.refresh.token"), "refresh token");
assertNotNull(conf.get("dfs.webhdfs.oauth2.refresh.url"), "refresh url");
assertNotNull(conf.get("dfs.webhdfs.oauth2.client.id"), "client id");

try {
  fs.open(p);
} catch (IOException e) {
  Throwable root = e.getCause() != null ? e.getCause() : e;
  if (root instanceof IllegalArgumentException && root.getMessage()
        .contains("Received invalid http response")) {
    // message embeds status + IdP body: invalid_grant -> refresh token,
    // invalid_client -> client id/credential, 5xx -> retry later
  }
}

When it happens

Trigger: refresh() executing (access token expired, so WebHdfsFileSystem fetches a new one) and the token endpoint returning non-200: expired/revoked refresh token (400 invalid_grant), wrong refresh URL, wrong client id, network appliance returning 401/403/502.

Common situations: Long-running jobs whose refresh token was revoked or expired mid-run; typo'd dfs.webhdfs.oauth2.refresh.url; Azure AD/Google IdP rejecting the request due to wrong resource/scope or redirected sign-in (HTML body); dev/prod config mix-ups where the credential belongs to another tenant.

Related errors

• Exception while refreshing access token
• Received invalid http response: {statusCode}, text = {text}
• Unable to load OAuth2 connection factory.
• Credential has not been provided in configuration
• Unable to obtain access token from credential

AI-assisted analysis of apache/hadoop@2add963021 (2026-08-22). Data as JSON: /api/errors/2285e86409cdba4b. Report an issue: GitHub.