{"record":{"id":"5e8b7ca625ac0b18","repo":"apache/hadoop","slug":"putimage-failed-stringutils-stringifyexceptio","errorCode":null,"errorMessage":"PutImage failed. \" + StringUtils.stringifyException(t)","messagePattern":"PutImage failed\\. \" \\+ StringUtils\\.stringifyException\\(t\\)","errorType":"http","errorClass":"IOException","httpStatus":410,"severity":"error","filePath":"hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/ImageServlet.java","lineNumber":700,"sourceCode":"                  nnImage.purgeOldStorage(nnf);\n                } finally {\n                  // remove the request once we've processed it, or it threw an error, so we\n                  // aren't using it either\n                  currentlyDownloadingCheckpoints.remove(imageRequest);\n\n                  stream.close();\n                }\n              } finally {\n                nnImage.removeFromCheckpointing(txid);\n              }\n              return null;\n            }\n\n          });\n    } catch (Throwable t) {\n      String errMsg = \"PutImage failed. \" + StringUtils.stringifyException(t);\n      sendError(response, HttpServletResponse.SC_GONE, errMsg);\n      throw new IOException(errMsg);\n    }\n  }\n\n  private void sendError(HttpServletResponse response, int code, String message)\n      throws IOException {\n    if (response instanceof Response) {\n      ((Response)response).setStatusWithReason(code, message);\n    }\n\n    response.sendError(code, message);\n  }\n\n  /*\n   * Params required to handle put image request\n   */\n  static class PutImageParams {\n    private long txId = -1;\n    private String storageInfoString = null;","sourceCodeStart":682,"sourceCodeEnd":718,"githubUrl":"https://github.com/apache/hadoop/blob/2add9630210752f88ceb1bb74eb65e37bf41da8e/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/ImageServlet.java#L682-L718","documentation":"The PUT counterpart of GetImage: ImageServlet.putImage wraps every Throwable raised while the NameNode receives and persists a checkpoint fsimage uploaded by the Secondary - it streams the image into FSImage storage inside a fenced callable, checks the announced size/txid and updates checkpointing state. On failure the peer gets HTTP 410 Gone with the stringified cause and the servlet rethrows IOException. As with GetImage, the actionable cause is the nested exception, not this wrapper.","triggerScenarios":"2NN uploads the image (PUT with txid/fileLength/storageinfo) and the NN fails to save it: disk full or name dir removed, MD5/size mismatch against the announced fileLength, the Checkpointer rejecting a stale txid, or an exception in the checkpoint-state management around nnImage.removeFromCheckpointing.","commonSituations":"Checkpoint name dirs full or read-only; NN storage reformatted while the 2NN kept old state; truncated uploads through proxies or flaky Kerberos HTTP channels; concurrent checkpoints from HA pairs racing the upload.","solutions":["Unwrap the nested stack in the 410 body / NN log - root causes range from DiskErrorException to EOFException on truncated upload.","Verify free space and write permission on every dfs.namenode.name.dir of the receiving NameNode.","On MD5/length mismatch, simply retry the checkpoint; if uploads keep truncating, inspect MTU, proxies and SPNEGO on the HTTP channel to port 9870.","Confirm the 2NN is checkpointing against the right namespace - a storage-info mismatch often surfaces here as a PutImage failure."],"exampleFix":null,"handlingStrategy":"try-catch","validationCode":"// Uploader side: verify the announced metadata before the PUT\nif (imageFile.length() <= 0 || txid < 0 || storageInfo == null || storageInfo.isEmpty()) {\n  throw new IllegalStateException(\"Refusing PUT: fileLength/txid/storageInfo incomplete\");\n}","typeGuard":null,"tryCatchPattern":"try {\n  secondary.uploadImageToActive(txid, imageFile); // 2NN push path\n} catch (IOException e) {\n  if (e.getMessage() != null && e.getMessage().startsWith(\"PutImage failed\")) {\n    LOG.warn(\"Upload rejected; root cause is nested - aborting this checkpoint cycle\");\n    scheduler.scheduleNextCheckpoint(); // fresh attempt with new txid\n    return;\n  }\n  throw e;\n}","preventionTips":["Keep headroom on every dfs.namenode.name.dir; disk-full is the most common root cause.","Align NN and 2NN versions so upload metadata (fileLength, imageType) matches the contract.","Watch the checkpoint metrics (LastCheckpointTime, transactions since checkpoint) to detect persistently failing uploads early."],"tags":["hdfs","namenode","checkpoint","fsimage","upload"],"backgroundTag":"fsimage-transfer-failed","analyzedSha":"2add9630210752f88ceb1bb74eb65e37bf41da8e","analyzedAt":"2026-08-22T19:55:07.957Z","schemaVersion":2},"datasetVersion":"2026-08-22T20:17:22.307Z"}