apache/hadoop · error · IOException

Content-Length header is not provided by the namenode when t

Error message

Content-Length header is not provided by the namenode when trying to fetch " + url

What it means

Util.doGetUrl downloads an fsimage/editlog segment from the NameNode's image transfer servlet over HTTP and requires the response to carry a Content-Length header so the transfer can be size-verified. If the header is absent the fetch aborts before any bytes are read, because the size and MD5 verification in receiveFile would be meaningless. The real GetImageServlet always sets Content-Length, so a missing header almost always means the URL did not reach the real servlet.

Source

Thrown at hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/common/Util.java:182

    } catch (AuthenticationException e) {
      throw new IOException(e);
    }

    setTimeout(connection, timeout);

    if (connection.getResponseCode() != HttpURLConnection.HTTP_OK) {
      throw new HttpGetFailedException("Image transfer servlet at " + url +
              " failed with status code " + connection.getResponseCode() +
              "\nResponse message:\n" + connection.getResponseMessage(),
          connection);
    }

    long advertisedSize;
    String contentLength = connection.getHeaderField(CONTENT_LENGTH);
    if (contentLength != null) {
      advertisedSize = Long.parseLong(contentLength);
    } else {
      throw new IOException(CONTENT_LENGTH + " header is not provided " +
          "by the namenode when trying to fetch " + url);
    }
    MD5Hash advertisedDigest = parseMD5Header(connection);
    String fsImageName = connection
        .getHeaderField(ImageServlet.HADOOP_IMAGE_EDITS_HEADER);
    InputStream stream = connection.getInputStream();

    return receiveFile(url.toExternalForm(), localPaths, dstStorage,
        getChecksum, advertisedSize, advertisedDigest, fsImageName, stream,
        throttler);
  }

  /**
   * Receives file at the url location from the input stream and puts them in
   * the specified destination storage location.
   */
  public static MD5Hash receiveFile(String url, List<File> localPaths,
      Storage dstStorage, boolean getChecksum, long advertisedSize,

View on GitHub (pinned to 2add963021)

Solutions

Run 'curl -sv -o /dev/null <url>' against the exact fetch URL from the 2NN/standby host and confirm which hop drops Content-Length
Point the checkpoint/http address (dfs.namenode.http-address, dfs.journalnode...) directly at the NameNode port, bypassing any proxy or LB
Verify both ends run the same Hadoop version and the URL path hits GetImageServlet (…/getimage), not a UI or error page
If a custom servlet wrapper or filter sits in front, make it set Content-Length instead of chunked transfer encoding

Defensive patterns

Strategy: try-catch

Validate before calling

// Preflight: confirm the endpoint will behave like GetImageServlet
HttpURLConnection c = (HttpURLConnection) new URL(imageFetchUrl).openConnection();
c.setRequestMethod("GET");
if (c.getResponseCode() != 200
    || c.getHeaderField("Content-Length") == null) {
  throw new IllegalStateException(
      "Endpoint will not serve a verifiable image: " + imageFetchUrl);
}

Try / catch

try {
  Util.doGetUrl(url, localPaths, storage, true, timeoutMs, throttler);
} catch (IOException e) {
  if (e.getMessage() != null && e.getMessage().contains("header is not provided")) {
    // endpoint/proxy problem, not transient — do not blind-retry; fix the URL path
    LOG.error("Fetch endpoint missing Content-Length: {}", url);
  }
  throw e;
}

Prevention

Point checkpoint/image-transfer config directly at the NameNode HTTP port; never through an LB that rewrites responses
Smoke-test the getimage URL with curl -I from every standby/2NN host during cluster bring-up
Keep NN and clients on matching Hadoop versions so the servlet contract holds

When it happens

Trigger: Calling Util.doGetUrl (the path taken by SecondaryNameNode/standby checkpoint transfer and bootstrapStandby image downloads) against a URL whose 200 OK response lacks Content-Length: a proxy or load balancer that strips/buffers the header, a chunked-encoded response from a custom servlet or filter, or an HTML 200 page served by an LB/KMS gateway on the wrong port.

Common situations: HTTP(S) proxy between 2NN/standby and the NN; SPNEGO through a gateway that rewrites responses; dfs.namenode.http-address or the checkpoint address pointing at a web UI / LB port instead of the NN's GetImageServlet endpoint; a forked/older Hadoop whose servlet streams chunked.

Related errors

AI-assisted analysis of apache/hadoop@2add963021 (2026-08-22). Data as JSON: /api/errors/84d2c37bb4a97b4b. Report an issue: GitHub.