{"record":{"id":"22cc01ce9132ea53","repo":"apache/hadoop","slug":"access-denied-dfs-http-policy-is-https-only","errorCode":null,"errorMessage":"Access denied: dfs.http.policy is HTTPS_ONLY.","messagePattern":"Access denied: dfs\\.http\\.policy is HTTPS_ONLY\\.","errorType":"exception","errorClass":"IOException","httpStatus":null,"severity":"critical","filePath":"hadoop-hdfs-project/hadoop-hdfs-client/src/main/java/org/apache/hadoop/hdfs/web/WebHdfsFileSystem.java","lineNumber":729,"sourceCode":"     * send out the data before the redirect. This issue is addressed by the\n     * \"Expect: 100-continue\" header in HTTP/1.1; see RFC 2616, Section 8.2.3.\n     * Unfortunately, there are software library bugs (e.g. Jetty 6 http server\n     * and Java 6 http client), which do not correctly implement \"Expect:\n     * 100-continue\". The two-step create/append is a temporary workaround for\n     * the software library bugs.\n     *\n     * Open/Checksum\n     * Also implements two-step connects for other operations redirected to\n     * a DN such as open and checksum\n     */\n    protected HttpURLConnection connect(URL url) throws IOException {\n      //redirect hostname and port\n      redirectHost = null;\n\n      if (url.getProtocol().equals(\"http\") &&\n        UserGroupInformation.isSecurityEnabled() &&\n        isTLSKrb) {\n        throw new IOException(\"Access denied: dfs.http.policy is HTTPS_ONLY.\");\n      }\n\n      // resolve redirects for a DN operation unless already resolved\n      if (op.getRedirect() && !redirected) {\n        final HttpOpParam.Op redirectOp =\n            HttpOpParam.TemporaryRedirectOp.valueOf(op);\n        final HttpURLConnection conn = connect(redirectOp, url);\n        // application level proxy like httpfs might not issue a redirect\n        if (conn.getResponseCode() == op.getExpectedHttpResponseCode()) {\n          return conn;\n        }\n        try {\n          validateResponse(redirectOp, conn, false);\n          url = new URL(conn.getHeaderField(\"Location\"));\n          redirectHost = url.getHost() + \":\" + url.getPort();\n        } finally {\n          // TODO: consider not calling conn.disconnect() to allow connection reuse\n          // See http://tinyurl.com/java7-http-keepalive","sourceCodeStart":711,"sourceCodeEnd":747,"githubUrl":"https://github.com/apache/hadoop/blob/2add9630210752f88ceb1bb74eb65e37bf41da8e/hadoop-hdfs-project/hadoop-hdfs-client/src/main/java/org/apache/hadoop/hdfs/web/WebHdfsFileSystem.java#L711-L747","documentation":"Before connecting, WebHdfsFileSystem checks whether the target URL uses plain http while Kerberos security is enabled and the client configuration says dfs.http.policy is HTTPS_ONLY. It deliberately refuses the connection so Kerberos-authenticated WebHDFS traffic is never downgraded to plaintext HTTP. This is a client-side policy enforcement check, not a response from the server.","triggerScenarios":"Initializing or using a FileSystem with a webhdfs:// URL when UserGroupInformation.isSecurityEnabled() is true and dfs.http.policy=HTTPS_ONLY in the client Configuration. The equivalent HTTPS URL with swebhdfs:// does not trigger it.","commonSituations":"A cluster is configured for HTTPS_ONLY but jobs or documentation still use webhdfs://; port copied from the HTTP endpoint; a security hardening change was not followed by client URL updates; client core-site.xml retains the server's HTTPS_ONLY policy while the code chooses http.","solutions":["Change the FileSystem URI from webhdfs://namenode:http-port to swebhdfs://namenode:https-port.","Ensure the client can trust the server certificate and has valid hadoop.ssl.client.conf / truststore settings.","Only if the security policy intentionally allows HTTP, change dfs.http.policy on the server and client to HTTP_AND_HTTPS or HTTP_ONLY; do not weaken HTTPS_ONLY merely to bypass the error.","Audit configuration and URLs together during a security-policy migration so scheme, port, and dfs.http.policy agree."],"exampleFix":"// before\nFileSystem fs = FileSystem.get(new URI(\"webhdfs://nn:9870\"), conf);\n\n// after\nFileSystem fs = FileSystem.get(new URI(\"swebhdfs://nn:9871\"), conf);","handlingStrategy":"validation","validationCode":"if (UserGroupInformation.isSecurityEnabled()\n    && \"HTTPS_ONLY\".equals(conf.get(\"dfs.http.policy\"))\n    && \"webhdfs\".equalsIgnoreCase(new Path(String.valueOf(url)).toUri().getScheme())) {\n  throw new IllegalArgumentException(\"Use swebhdfs:// because dfs.http.policy is HTTPS_ONLY\");\n}","typeGuard":null,"tryCatchPattern":"try {\n  return FileSystem.get(new URI(\"webhdfs://nn:9870\"), conf);\n} catch (IOException e) {\n  if (\"Access denied: dfs.http.policy is HTTPS_ONLY.\".equals(e.getMessage())) {\n    return FileSystem.get(new URI(\"swebhdfs://nn:9871\"), conf);\n  }\n  throw e;\n}","preventionTips":["Choose webhdfs:// versus swebhdfs:// from the configured dfs.http.policy in deployment templates.","Never downgrade HTTPS_ONLY policy to send Kerberos-authenticated traffic over HTTP.","Validate client truststore and SSL configuration when switching to swebhdfs://."],"tags":["java","hadoop","webhdfs","security","kerberos","https","configuration"],"backgroundTag":"https-required","analyzedSha":"2add9630210752f88ceb1bb74eb65e37bf41da8e","analyzedAt":"2026-08-22T19:55:07.957Z","schemaVersion":2},"datasetVersion":"2026-08-22T20:17:22.307Z"}