apache/hadoop · error · IOException

File " + url + " computed digest " + computedDigest + " does

Error message

File " + url + " computed digest " + computedDigest + " does not match advertised digest " + advertisedDigest

What it means

When checksum verification is requested, receiveFile MD5-digests the downloaded bytes and compares against the digest the NN advertised in the response headers. A mismatch means the bytes written are not the bytes the NN intended — corruption in transit or on write. Temp files are deleted before throwing.

Source

Thrown at hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/common/Util.java:327

        // -- otherwise a client-side IOException would be masked by this
        // exception that makes it look like a server-side problem!
        deleteTmpFiles(localPaths);
        throw new IOException("File " + url + " received length " + received +
            " is not of the advertised size " + advertisedSize +
            ". Fsimage name: " + fsImageName + " lastReceived: " + num);
      }
    }
    xferStats.insert(0, String.format("Combined time for file download and" +
        " fsync to all disks took %.2fs.", xferCombined));
    LOG.info(xferStats.toString());

    if (digester != null) {
      MD5Hash computedDigest = new MD5Hash(digester.digest());

      if (advertisedDigest != null &&
          !computedDigest.equals(advertisedDigest)) {
        deleteTmpFiles(localPaths);
        throw new IOException("File " + url + " computed digest " +
            computedDigest + " does not match advertised digest " +
            advertisedDigest);
      }
      return computedDigest;
    } else {
      return null;
    }
  }

  private static void deleteTmpFiles(List<File> files) {
    if (files == null) {
      return;
    }

    LOG.info("Deleting temporary files: " + files);
    for (File file : files) {
      if (!file.delete()) {
        LOG.warn("Deleting " + file + " has failed");

View on GitHub (pinned to 2add963021)

Solutions

1. Retry the transfer once — transient corruption is the most common cause and temp files were already removed
2. If it recurs on one host, run hardware diagnostics (smartctl, memtest) and check NIC/switch error counters; disable RX/TX offload on suspect NICs
3. Manually compare MD5 of the image on the NN vs a manually downloaded copy to localize corruption to sender, path, or receiver
4. Verify no concurrent checkpoint is mutating the file being served

try {
  Util.doGetUrl(url, localPaths, storage, true, timeoutMs, throttler);
} catch (IOException e) {
  if (e.getMessage() != null && e.getMessage().contains("does not match advertised digest")) {
    // corrupted in transit or on write; temp files removed — one retry is cheap
    return retryOnce();
  }
  throw e;
}

When it happens

Trigger: Silent bit corruption on the network path (flaky NIC/cable/switch, offload engine bugs), a proxy rewriting the body, a failing disk on the receiving node corrupting written data, or reading a file that changed concurrently on the NN between digest computation and streaming.

Common situations: One host in the cluster repeatedly failing image transfers (bad RAM/disk); transfers that pass a size check but fail MD5 after maintenance on switching gear; oversized MTU/Jumbo-frame misconfigurations.

Related errors

• Content-Length header is not provided by the namenode when t
• No filename header provided by server
• Checksum failed at {failedPos} for replica: {replica}
• Directory {dir} is in an inconsistent state: Message digest
• serial id ${id} > ${maxEntryNumber}

AI-assisted analysis of apache/hadoop@2add963021 (2026-08-22). Data as JSON: /api/errors/f9d17dd362b7a989. Report an issue: GitHub.