{"record":{"id":"2285e86409cdba4b","repo":"apache/hadoop","slug":"received-invalid-http-response-statuscode-text","errorCode":null,"errorMessage":"Received invalid http response: {statusCode}, text = {text}","messagePattern":"Received invalid http response: (.+?), text = (.+?)","errorType":"exception","errorClass":"IllegalArgumentException","httpStatus":null,"severity":"error","filePath":"hadoop-hdfs-project/hadoop-hdfs-client/src/main/java/org/apache/hadoop/hdfs/web/oauth2/ConfRefreshTokenBasedAccessTokenProvider.java","lineNumber":130,"sourceCode":"  void refresh() throws IOException {\n    final List<NameValuePair> pairs = new ArrayList<>();\n    pairs.add(new BasicNameValuePair(GRANT_TYPE, REFRESH_TOKEN));\n    pairs.add(new BasicNameValuePair(REFRESH_TOKEN, refreshToken));\n    pairs.add(new BasicNameValuePair(CLIENT_ID, clientId));\n    final RequestConfig config = RequestConfig.custom()\n        .setConnectTimeout(URLConnectionFactory.DEFAULT_SOCKET_TIMEOUT)\n        .setConnectionRequestTimeout(URLConnectionFactory.DEFAULT_SOCKET_TIMEOUT)\n        .setSocketTimeout(URLConnectionFactory.DEFAULT_SOCKET_TIMEOUT)\n        .build();\n    try (CloseableHttpClient client =\n             HttpClientBuilder.create().setDefaultRequestConfig(config).build()) {\n      final HttpPost httpPost = new HttpPost(refreshURL);\n      httpPost.setEntity(new UrlEncodedFormEntity(pairs, StandardCharsets.UTF_8));\n      httpPost.setHeader(HttpHeaders.CONTENT_TYPE, URLENCODED);\n      try (CloseableHttpResponse response = client.execute(httpPost)) {\n        final int statusCode = response.getStatusLine().getStatusCode();\n        if (statusCode != HttpStatus.SC_OK) {\n          throw new IllegalArgumentException(\n              \"Received invalid http response: \" + statusCode + \", text = \" +\n                  EntityUtils.toString(response.getEntity()));\n        }\n        Map<?, ?> responseBody = JsonSerialization.mapReader().readValue(\n            EntityUtils.toString(response.getEntity()));\n\n        String newExpiresIn = responseBody.get(EXPIRES_IN).toString();\n        accessTokenTimer.setExpiresIn(newExpiresIn);\n\n        accessToken = responseBody.get(ACCESS_TOKEN).toString();\n      }\n    } catch (RuntimeException e) {\n      throw new IOException(\"Exception while refreshing access token\", e);\n    } catch (Exception e) {\n      throw new IOException(\"Exception while refreshing access token\", e);\n    }\n  }\n","sourceCodeStart":112,"sourceCodeEnd":148,"githubUrl":"https://github.com/apache/hadoop/blob/2add9630210752f88ceb1bb74eb65e37bf41da8e/hadoop-hdfs-project/hadoop-hdfs-client/src/main/java/org/apache/hadoop/hdfs/web/oauth2/ConfRefreshTokenBasedAccessTokenProvider.java#L112-L148","documentation":"ConfRefreshTokenBasedAccessTokenProvider.refresh posts a refresh_token grant to dfs.webhdfs.oauth2.refresh.url; any HTTP status other than 200 throws IllegalArgumentException embedding the status code and the raw response body. The body text is the key diagnostic — OAuth2 errors like invalid_grant (expired/revoked refresh token), invalid_client (bad dfs.webhdfs.oauth2.client.id), or 5xx from the identity provider appear there verbatim.","triggerScenarios":"refresh() executing (access token expired, so WebHdfsFileSystem fetches a new one) and the token endpoint returning non-200: expired/revoked refresh token (400 invalid_grant), wrong refresh URL, wrong client id, network appliance returning 401/403/502.","commonSituations":"Long-running jobs whose refresh token was revoked or expired mid-run; typo'd dfs.webhdfs.oauth2.refresh.url; Azure AD/Google IdP rejecting the request due to wrong resource/scope or redirected sign-in (HTML body); dev/prod config mix-ups where the credential belongs to another tenant.","solutions":["Read the embedded response text in the exception message: invalid_grant means obtain and configure a fresh refresh token in dfs.webhdfs.oauth2.refresh.token","Verify dfs.webhdfs.oauth2.refresh.url points at the real token endpoint (e.g. https://login.microsoftonline.com/<tenant>/oauth2/token) and dfs.webhdfs.oauth2.client.id matches the registered application","Reproduce outside Hadoop: curl -d 'grant_type=refresh_token&refresh_token=...' ... to see the exact IdP error","If the body is HTML, you are hitting a login/redirect page — fix the URL; if 5xx, the IdP is down, retry later"],"exampleFix":"<!-- before: stale refresh token in core-site.xml -->\n<property><name>dfs.webhdfs.oauth2.refresh.token</name><value>old-token</value></property>\n<!-- after: replace with the token freshly issued by the IdP -->\n<property><name>dfs.webhdfs.oauth2.refresh.token</name><value>AQEAA...</value></property>","handlingStrategy":"try-catch","validationCode":"// No safe pre-check of the IdP exists; validate local config instead\nassertNotNull(conf.get(\"dfs.webhdfs.oauth2.refresh.token\"), \"refresh token\");\nassertNotNull(conf.get(\"dfs.webhdfs.oauth2.refresh.url\"), \"refresh url\");\nassertNotNull(conf.get(\"dfs.webhdfs.oauth2.client.id\"), \"client id\");","typeGuard":null,"tryCatchPattern":"try {\n  fs.open(p);\n} catch (IOException e) {\n  Throwable root = e.getCause() != null ? e.getCause() : e;\n  if (root instanceof IllegalArgumentException && root.getMessage()\n        .contains(\"Received invalid http response\")) {\n    // message embeds status + IdP body: invalid_grant -> refresh token,\n    // invalid_client -> client id/credential, 5xx -> retry later\n  }\n}","preventionTips":["Automate refresh-token rotation before expiry so invalid_grant never happens in production","Validate the token endpoint with curl during deployment","Treat any change to dfs.webhdfs.oauth2.* as requiring an end-to-end OAuth smoke test"],"tags":["oauth2","webhdfs","http-status","refresh-token"],"backgroundTag":"oauth2-token-refresh-failed","analyzedSha":"2add9630210752f88ceb1bb74eb65e37bf41da8e","analyzedAt":"2026-08-22T19:55:07.957Z","schemaVersion":2},"datasetVersion":"2026-08-22T20:17:22.307Z"}