{"record":{"id":"a82b2eff3da8d90c","repo":"apache/hadoop","slug":"missing-http-location-header-for-0","errorCode":null,"errorMessage":"Missing HTTP 'Location' header for [{0}]","messagePattern":"Missing HTTP 'Location' header for \\[(.+?)\\]","errorType":"exception","errorClass":"IOException","httpStatus":null,"severity":"error","filePath":"hadoop-hdfs-project/hadoop-hdfs-httpfs/src/main/java/org/apache/hadoop/fs/http/client/HttpFSFileSystem.java","lineNumber":590,"sourceCode":"    conn.setInstanceFollowRedirects(false);\n    boolean exceptionAlreadyHandled = false;\n    try {\n      if (conn.getResponseCode() == HTTP_TEMPORARY_REDIRECT) {\n        exceptionAlreadyHandled = true;\n        String location = conn.getHeaderField(\"Location\");\n        if (location != null) {\n          conn = getConnection(new URL(location), method);\n          conn.setRequestProperty(\"Content-Type\", UPLOAD_CONTENT_TYPE);\n          try {\n            OutputStream os = new BufferedOutputStream(conn.getOutputStream(), bufferSize);\n            return new HttpFSDataOutputStream(conn, os, expectedStatus, statistics);\n          } catch (IOException ex) {\n            HttpExceptionUtils.validateResponse(conn, expectedStatus);\n            throw ex;\n          }\n        } else {\n          HttpExceptionUtils.validateResponse(conn, HTTP_TEMPORARY_REDIRECT);\n          throw new IOException(\"Missing HTTP 'Location' header for [\" + conn.getURL() + \"]\");\n        }\n      } else {\n        throw new IOException(\n          MessageFormat.format(\"Expected HTTP status was [307], received [{0}]\",\n                               conn.getResponseCode()));\n      }\n    } catch (IOException ex) {\n      if (exceptionAlreadyHandled) {\n        throw ex;\n      } else {\n        HttpExceptionUtils.validateResponse(conn, HTTP_TEMPORARY_REDIRECT);\n        throw ex;\n      }\n    }\n  }\n\n\n  /**","sourceCodeStart":572,"sourceCodeEnd":608,"githubUrl":"https://github.com/apache/hadoop/blob/2add9630210752f88ceb1bb74eb65e37bf41da8e/hadoop-hdfs-project/hadoop-hdfs-httpfs/src/main/java/org/apache/hadoop/fs/http/client/HttpFSFileSystem.java#L572-L608","documentation":"During WebHDFS/HttpFS create() and append(), the client first receives HTTP 307 Temporary Redirect and then expects a 'Location' header naming the data node (or HttpFS data endpoint) to stream bytes to (HttpFSFileSystem.java:573-594). If the 307 arrives without a Location header, validateResponse runs and this IOException is thrown naming the connection URL. It almost always means something between client and server — a reverse proxy, gateway, or non-WebHDFS service — emitted or rewrote the 307 rather than the HttpFS/WebHDFS endpoint itself.","triggerScenarios":"Nginx/haproxy/ALB in front of HttpFS (port 14000) answering or rewriting 307s with proxy_redirect/redirect directives that drop Location; Knox or API-gateway versions that mishandle WebHDFS redirects for the data phase; the create/append URL pointing at a service that returns 307 without Location (e.g. an auth portal redirect captured before status validation); curl-level tests where an HTTP-to-HTTPS redirector strips the header.","commonSituations":"Corporate TLS terminators proxying webhdfs:// traffic; HttpFS behind Knox with outdated gateway config; switching clients from direct NameNode WebHDFS (9870) to an HttpFS URL and hitting an intermediary for the first time; noredirect=false flows where the data phase still expects the redirect to the HttpFS host itself.","solutions":["Reproduce with curl -i on the exact create/append URL: the 307 must carry a Location header; if absent, the intermediary is the culprit.","Fix the proxy: disable/limit redirect rewriting (nginx: proxy_redirect off; do not intercept 3xx), and pass Location through unmodified; or bypass the proxy for the HttpFS data endpoints.","If a gateway like Knox is used, upgrade/verify its WebHDFS dispatch so it forwards the datanode/HttpFS Location for the write phase.","As a last resort point the client straight at the WebHDFS/HttpFS endpoint (webhdfs://httpfs-host:14000) to confirm the header exists without middleboxes."],"exampleFix":"# before (nginx strips/rewrites the redirect)\nlocation /webhdfs { proxy_pass http://httpfs:14000; proxy_redirect http://https://; }\n# after\nlocation /webhdfs { proxy_pass http://httpfs:14000; proxy_redirect off; proxy_set_header Host $host; }\n# verify: curl -i -X PUT 'http://gw/webhdfs/v1/f?op=CREATE&noredirect=false' must show Location:","handlingStrategy":"try-catch","validationCode":null,"typeGuard":null,"tryCatchPattern":"try (FSDataOutputStream out = fs.create(path, true)) {\n  out.write(data);\n} catch (IOException e) {\n  if (e.getMessage() != null && e.getMessage().contains(\"Missing HTTP 'Location' header\")) {\n    throw new IOException(\"WebHDFS/HttpFS 307 redirect lost its Location header — \"\n        + \"check reverse proxy / gateway rewrite rules for \" + fs.getUri(), e);\n  }\n  throw e;\n}","preventionTips":["Smoke-test write paths with curl -i and assert the 307 carries a Location header before going live behind a proxy.","Configure nginx with proxy_redirect off and never intercept 3xx for /webhdfs locations.","Keep gateway (Knox/proxy) versions known-good for WebHDFS data-phase redirects; test create and append after every upgrade.","Expose fs.defaultFS as webhdfs://<httpfs>:14000 directly for diagnostic runs to isolate middleboxes."],"tags":["webhdfs","httpfs","http-redirect","proxy","io"],"backgroundTag":"missing-redirect-location-header","analyzedSha":"2add9630210752f88ceb1bb74eb65e37bf41da8e","analyzedAt":"2026-08-22T19:55:07.957Z","schemaVersion":2},"datasetVersion":"2026-08-22T20:17:22.307Z"}