{"record":{"id":"84d2c37bb4a97b4b","repo":"apache/hadoop","slug":"content-length-header-is-not-provided-by-the-namen","errorCode":null,"errorMessage":"Content-Length header is not provided by the namenode when trying to fetch \" + url","messagePattern":"Content-Length header is not provided by the namenode when trying to fetch \" \\+ url","errorType":"exception","errorClass":"IOException","httpStatus":null,"severity":"error","filePath":"hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/common/Util.java","lineNumber":182,"sourceCode":"    } catch (AuthenticationException e) {\n      throw new IOException(e);\n    }\n\n    setTimeout(connection, timeout);\n\n    if (connection.getResponseCode() != HttpURLConnection.HTTP_OK) {\n      throw new HttpGetFailedException(\"Image transfer servlet at \" + url +\n              \" failed with status code \" + connection.getResponseCode() +\n              \"\\nResponse message:\\n\" + connection.getResponseMessage(),\n          connection);\n    }\n\n    long advertisedSize;\n    String contentLength = connection.getHeaderField(CONTENT_LENGTH);\n    if (contentLength != null) {\n      advertisedSize = Long.parseLong(contentLength);\n    } else {\n      throw new IOException(CONTENT_LENGTH + \" header is not provided \" +\n          \"by the namenode when trying to fetch \" + url);\n    }\n    MD5Hash advertisedDigest = parseMD5Header(connection);\n    String fsImageName = connection\n        .getHeaderField(ImageServlet.HADOOP_IMAGE_EDITS_HEADER);\n    InputStream stream = connection.getInputStream();\n\n    return receiveFile(url.toExternalForm(), localPaths, dstStorage,\n        getChecksum, advertisedSize, advertisedDigest, fsImageName, stream,\n        throttler);\n  }\n\n  /**\n   * Receives file at the url location from the input stream and puts them in\n   * the specified destination storage location.\n   */\n  public static MD5Hash receiveFile(String url, List<File> localPaths,\n      Storage dstStorage, boolean getChecksum, long advertisedSize,","sourceCodeStart":164,"sourceCodeEnd":200,"githubUrl":"https://github.com/apache/hadoop/blob/2add9630210752f88ceb1bb74eb65e37bf41da8e/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/common/Util.java#L164-L200","documentation":"Util.doGetUrl downloads an fsimage/editlog segment from the NameNode's image transfer servlet over HTTP and requires the response to carry a Content-Length header so the transfer can be size-verified. If the header is absent the fetch aborts before any bytes are read, because the size and MD5 verification in receiveFile would be meaningless. The real GetImageServlet always sets Content-Length, so a missing header almost always means the URL did not reach the real servlet.","triggerScenarios":"Calling Util.doGetUrl (the path taken by SecondaryNameNode/standby checkpoint transfer and bootstrapStandby image downloads) against a URL whose 200 OK response lacks Content-Length: a proxy or load balancer that strips/buffers the header, a chunked-encoded response from a custom servlet or filter, or an HTML 200 page served by an LB/KMS gateway on the wrong port.","commonSituations":"HTTP(S) proxy between 2NN/standby and the NN; SPNEGO through a gateway that rewrites responses; dfs.namenode.http-address or the checkpoint address pointing at a web UI / LB port instead of the NN's GetImageServlet endpoint; a forked/older Hadoop whose servlet streams chunked.","solutions":["Run 'curl -sv -o /dev/null <url>' against the exact fetch URL from the 2NN/standby host and confirm which hop drops Content-Length","Point the checkpoint/http address (dfs.namenode.http-address, dfs.journalnode...) directly at the NameNode port, bypassing any proxy or LB","Verify both ends run the same Hadoop version and the URL path hits GetImageServlet (…/getimage), not a UI or error page","If a custom servlet wrapper or filter sits in front, make it set Content-Length instead of chunked transfer encoding"],"exampleFix":null,"handlingStrategy":"try-catch","validationCode":"// Preflight: confirm the endpoint will behave like GetImageServlet\nHttpURLConnection c = (HttpURLConnection) new URL(imageFetchUrl).openConnection();\nc.setRequestMethod(\"GET\");\nif (c.getResponseCode() != 200\n    || c.getHeaderField(\"Content-Length\") == null) {\n  throw new IllegalStateException(\n      \"Endpoint will not serve a verifiable image: \" + imageFetchUrl);\n}","typeGuard":null,"tryCatchPattern":"try {\n  Util.doGetUrl(url, localPaths, storage, true, timeoutMs, throttler);\n} catch (IOException e) {\n  if (e.getMessage() != null && e.getMessage().contains(\"header is not provided\")) {\n    // endpoint/proxy problem, not transient — do not blind-retry; fix the URL path\n    LOG.error(\"Fetch endpoint missing Content-Length: {}\", url);\n  }\n  throw e;\n}","preventionTips":["Point checkpoint/image-transfer config directly at the NameNode HTTP port; never through an LB that rewrites responses","Smoke-test the getimage URL with curl -I from every standby/2NN host during cluster bring-up","Keep NN and clients on matching Hadoop versions so the servlet contract holds"],"tags":["hdfs","fsimage","checkpoint","http-header","namenode","transfer"],"backgroundTag":"missing-http-header","analyzedSha":"2add9630210752f88ceb1bb74eb65e37bf41da8e","analyzedAt":"2026-08-22T19:55:07.957Z","schemaVersion":2},"datasetVersion":"2026-08-23T01:17:44.959Z"}