{"record":{"id":"ca1d6bf8d663a4dd","repo":"apache/hadoop","slug":"file-url-received-length-received","errorCode":null,"errorMessage":"File \" + url + \" received length \" + received + \" is not of the advertised size \" + advertisedSize + \". Fsimage name: \" + fsImageName + \" lastReceived: \" + num","messagePattern":"File \" \\+ url \\+ \" received length \" \\+ received \\+ \" is not of the advertised size \" \\+ advertisedSize \\+ \"\\. Fsimage name: \" \\+ fsImageName \\+ \" lastReceived: \" \\+ num","errorType":"exception","errorClass":"IOException","httpStatus":null,"severity":"error","filePath":"hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/common/Util.java","lineNumber":312,"sourceCode":"        xferCombined += writeSec;\n        xferStats.append(String\n            .format(\" Synchronous (fsync) write to disk of \" +\n                streamPathMap.get(fos).getAbsolutePath() +\n                \" took %.2fs.\", writeSec));\n      }\n\n      // Something went wrong and did not finish reading.\n      // Remove the temporary files.\n      if (!finishedReceiving) {\n        deleteTmpFiles(localPaths);\n      }\n\n      if (finishedReceiving && received != advertisedSize) {\n        // only throw this exception if we think we read all of it on our end\n        // -- otherwise a client-side IOException would be masked by this\n        // exception that makes it look like a server-side problem!\n        deleteTmpFiles(localPaths);\n        throw new IOException(\"File \" + url + \" received length \" + received +\n            \" is not of the advertised size \" + advertisedSize +\n            \". Fsimage name: \" + fsImageName + \" lastReceived: \" + num);\n      }\n    }\n    xferStats.insert(0, String.format(\"Combined time for file download and\" +\n        \" fsync to all disks took %.2fs.\", xferCombined));\n    LOG.info(xferStats.toString());\n\n    if (digester != null) {\n      MD5Hash computedDigest = new MD5Hash(digester.digest());\n\n      if (advertisedDigest != null &&\n          !computedDigest.equals(advertisedDigest)) {\n        deleteTmpFiles(localPaths);\n        throw new IOException(\"File \" + url + \" computed digest \" +\n            computedDigest + \" does not match advertised digest \" +\n            advertisedDigest);\n      }","sourceCodeStart":294,"sourceCodeEnd":330,"githubUrl":"https://github.com/apache/hadoop/blob/2add9630210752f88ceb1bb74eb65e37bf41da8e/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/common/Util.java#L294-L330","documentation":"receiveFile counts bytes until the stream signals EOF; if the loop completed cleanly (finishedReceiving) but the byte count differs from the Content-Length advertised by the NameNode, the temp files are deleted and this mismatch is thrown. The finishedReceiving guard means a client-side read exception is never masked — this fires only when the server side delivered fewer bytes than promised with a clean end of stream.","triggerScenarios":"The NN (or an intermediary) closes the connection early but cleanly: NN crashing or being restarted mid-transfer, a proxy/LB truncating a large fsimage body on an idle/max-body-size timeout, or a concurrent checkpoint replacing the file while it streams.","commonSituations":"Large images (tens of GB) cut off by an LB with a response-size cap; NN failover during the 2NN/standby download; flaky middleboxes that close streams without RST.","solutions":["Retry the checkpoint/download — the temp files are already cleaned up and the transfer is idempotent","Raise or remove proxy/LB max-response-size and idle timeouts on the NN HTTP path","Check NameNode logs around the transfer time for a dying servlet thread or failover","Confirm the image on the NN has a stable size (no concurrent checkpoint writing it) before retrying"],"exampleFix":null,"handlingStrategy":"retry","validationCode":null,"typeGuard":null,"tryCatchPattern":"int attempts = 0;\nwhile (true) {\n  try {\n    Util.doGetUrl(url, localPaths, storage, true, timeoutMs, throttler);\n    break;\n  } catch (IOException e) {\n    boolean truncated = e.getMessage() != null\n        && e.getMessage().contains(\"is not of the advertised size\");\n    if (!truncated || ++attempts >= 3) throw e;\n    // temp files already removed; safe to retry after backoff\n    Thread.sleep(attempts * 5000L);\n  }\n}","preventionTips":["Size LB/proxy body and idle timeouts above your largest expected fsimage","Avoid restarting the NN or triggering checkpoints concurrently with image downloads","Watch image size growth so capacity planning keeps transfers short"],"tags":["hdfs","fsimage","truncated-download","content-length","checkpoint"],"backgroundTag":"truncated-download","analyzedSha":"2add9630210752f88ceb1bb74eb65e37bf41da8e","analyzedAt":"2026-08-22T19:55:07.957Z","schemaVersion":2},"datasetVersion":"2026-08-22T20:17:22.307Z"}