apache/hadoop · error · IOException

Exception while refreshing access token

Error message

Exception while refreshing access token

What it means

ConfRefreshTokenBasedAccessTokenProvider.refresh wraps any RuntimeException raised while refreshing the access token into IOException('Exception while refreshing access token') (WebHdfsFileSystem oauth2 class, line 143). The cause is attached, so getCause() holds the real failure. Typical causes: NullPointerException when the token endpoint response JSON lacks expires_in or access_token; IllegalStateException from AccessTokenTimer.setExpiresIn on a malformed expires_in; JSON mapping errors.

Source

Thrown at hadoop-hdfs-project/hadoop-hdfs-client/src/main/java/org/apache/hadoop/hdfs/web/oauth2/ConfRefreshTokenBasedAccessTokenProvider.java:143

      httpPost.setEntity(new UrlEncodedFormEntity(pairs, StandardCharsets.UTF_8));
      httpPost.setHeader(HttpHeaders.CONTENT_TYPE, URLENCODED);
      try (CloseableHttpResponse response = client.execute(httpPost)) {
        final int statusCode = response.getStatusLine().getStatusCode();
        if (statusCode != HttpStatus.SC_OK) {
          throw new IllegalArgumentException(
              "Received invalid http response: " + statusCode + ", text = " +
                  EntityUtils.toString(response.getEntity()));
        }
        Map<?, ?> responseBody = JsonSerialization.mapReader().readValue(
            EntityUtils.toString(response.getEntity()));

        String newExpiresIn = responseBody.get(EXPIRES_IN).toString();
        accessTokenTimer.setExpiresIn(newExpiresIn);

        accessToken = responseBody.get(ACCESS_TOKEN).toString();
      }
    } catch (RuntimeException e) {
      throw new IOException("Exception while refreshing access token", e);
    } catch (Exception e) {
      throw new IOException("Exception while refreshing access token", e);
    }
  }

  public String getRefreshToken() {
    return refreshToken;
  }
}

View on GitHub (pinned to 2add963021)

Solutions

1. Inspect e.getCause() and its message — it names the exact parse/validation failure
2. Capture the raw token endpoint response (curl the refresh endpoint with the same form fields) and confirm it contains access_token and expires_in
3. If the IdP omits expires_in, use a provider/endpoint configuration that returns it, or a custom AccessTokenProvider
4. If a proxy mangles responses, bypass or fix the proxy for the token URL

Example fix

// before
catch (IOException e) {
  LOG.error("refresh failed", e); // cause hidden in logs
}
// after: surface the root cause explicitly
if (e.getCause() != null) {
  LOG.error("refresh failed: {} / {}", e.getCause().getClass().getName(), e.getCause().getMessage());
}

try {
  return fs.open(p);
} catch (IOException e) {
  if ("Exception while refreshing access token".equals(e.getMessage())
      && transientCause(e.getCause())) {   // e.g. network IOException
    sleepBackoff();
    return fs.open(p);                     // provider refreshes again
  }
  throw e;
}

When it happens

Trigger: refresh() succeeds at the HTTP level (200) but response parsing fails: body is valid JSON without the expected fields, body is not JSON at all (HTML from a proxy), or expires_in has an unexpected format so accessTokenTimer.setExpiresIn throws.

Common situations: Token endpoint returns an error payload with 200; proxies injecting HTML error pages; IdPs that omit expires_in on refresh responses; clock/format mismatches in expires_in (seconds vs epoch ms) tripping the timer validation.

Related errors

• Received invalid http response: {statusCode}, text = {text}
• Unable to obtain access token from credential
• Unable to load OAuth2 connection factory.
• Credential has not been provided in configuration
• Received invalid http response: {statusCode}, text = {text}

AI-assisted analysis of apache/hadoop@2add963021 (2026-08-22). Data as JSON: /api/errors/a405b57aa7787daa. Report an issue: GitHub.