{"record":{"id":"f9d17dd362b7a989","repo":"apache/hadoop","slug":"file-url-computed-digest-computeddiges","errorCode":null,"errorMessage":"File \" + url + \" computed digest \" + computedDigest + \" does not match advertised digest \" + advertisedDigest","messagePattern":"File \" \\+ url \\+ \" computed digest \" \\+ computedDigest \\+ \" does not match advertised digest \" \\+ advertisedDigest","errorType":"exception","errorClass":"IOException","httpStatus":null,"severity":"error","filePath":"hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/common/Util.java","lineNumber":327,"sourceCode":"        // -- otherwise a client-side IOException would be masked by this\n        // exception that makes it look like a server-side problem!\n        deleteTmpFiles(localPaths);\n        throw new IOException(\"File \" + url + \" received length \" + received +\n            \" is not of the advertised size \" + advertisedSize +\n            \". Fsimage name: \" + fsImageName + \" lastReceived: \" + num);\n      }\n    }\n    xferStats.insert(0, String.format(\"Combined time for file download and\" +\n        \" fsync to all disks took %.2fs.\", xferCombined));\n    LOG.info(xferStats.toString());\n\n    if (digester != null) {\n      MD5Hash computedDigest = new MD5Hash(digester.digest());\n\n      if (advertisedDigest != null &&\n          !computedDigest.equals(advertisedDigest)) {\n        deleteTmpFiles(localPaths);\n        throw new IOException(\"File \" + url + \" computed digest \" +\n            computedDigest + \" does not match advertised digest \" +\n            advertisedDigest);\n      }\n      return computedDigest;\n    } else {\n      return null;\n    }\n  }\n\n  private static void deleteTmpFiles(List<File> files) {\n    if (files == null) {\n      return;\n    }\n\n    LOG.info(\"Deleting temporary files: \" + files);\n    for (File file : files) {\n      if (!file.delete()) {\n        LOG.warn(\"Deleting \" + file + \" has failed\");","sourceCodeStart":309,"sourceCodeEnd":345,"githubUrl":"https://github.com/apache/hadoop/blob/2add9630210752f88ceb1bb74eb65e37bf41da8e/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/common/Util.java#L309-L345","documentation":"When checksum verification is requested, receiveFile MD5-digests the downloaded bytes and compares against the digest the NN advertised in the response headers. A mismatch means the bytes written are not the bytes the NN intended — corruption in transit or on write. Temp files are deleted before throwing.","triggerScenarios":"Silent bit corruption on the network path (flaky NIC/cable/switch, offload engine bugs), a proxy rewriting the body, a failing disk on the receiving node corrupting written data, or reading a file that changed concurrently on the NN between digest computation and streaming.","commonSituations":"One host in the cluster repeatedly failing image transfers (bad RAM/disk); transfers that pass a size check but fail MD5 after maintenance on switching gear; oversized MTU/Jumbo-frame misconfigurations.","solutions":["Retry the transfer once — transient corruption is the most common cause and temp files were already removed","If it recurs on one host, run hardware diagnostics (smartctl, memtest) and check NIC/switch error counters; disable RX/TX offload on suspect NICs","Manually compare MD5 of the image on the NN vs a manually downloaded copy to localize corruption to sender, path, or receiver","Verify no concurrent checkpoint is mutating the file being served"],"exampleFix":null,"handlingStrategy":"retry","validationCode":null,"typeGuard":null,"tryCatchPattern":"try {\n  Util.doGetUrl(url, localPaths, storage, true, timeoutMs, throttler);\n} catch (IOException e) {\n  if (e.getMessage() != null && e.getMessage().contains(\"does not match advertised digest\")) {\n    // corrupted in transit or on write; temp files removed — one retry is cheap\n    return retryOnce();\n  }\n  throw e;\n}","preventionTips":["Track per-host transfer failure rates; recurring MD5 failures on one host point to hardware","Check NIC/switch error counters after any digest mismatch","Keep checksum verification enabled (getChecksum=true) so corruption is caught, not silently persisted"],"tags":["hdfs","fsimage","md5","checksum","data-corruption","transfer"],"backgroundTag":"checksum-mismatch","analyzedSha":"2add9630210752f88ceb1bb74eb65e37bf41da8e","analyzedAt":"2026-08-22T19:55:07.957Z","schemaVersion":2},"datasetVersion":"2026-08-23T01:17:44.959Z"}