{"record":{"id":"b4ba9a4de4743388","repo":"apache/hadoop","slug":"image-uploading-failed-status-d-url-s-messa","errorCode":null,"errorMessage":"Image uploading failed, status: %d, url: %s, message: %s","messagePattern":"Image uploading failed, status: (.+?), url: (.+?), message: (.+?)","errorType":"exception","errorClass":"HttpPutFailedException","httpStatus":null,"severity":"error","filePath":"hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/TransferFsImage.java","lineNumber":318,"sourceCode":"      if (imageFile.length() > chunkSize) {\n        // using chunked streaming mode to support upload of 2GB+ files and to\n        // avoid internal buffering.\n        // this mode should be used only if more than chunkSize data is present\n        // to upload. otherwise upload may not happen sometimes.\n        connection.setChunkedStreamingMode(chunkSize);\n      }\n\n      setTimeout(connection);\n\n      // set headers for verification\n      ImageServlet.setVerificationHeadersForPut(connection, imageFile);\n\n      // Write the file to output stream.\n      writeFileToPutRequest(conf, connection, imageFile, canceler, chunkSize);\n\n      int responseCode = connection.getResponseCode();\n      if (responseCode != HttpURLConnection.HTTP_OK) {\n        throw new HttpPutFailedException(String.format(\n            \"Image uploading failed, status: %d, url: %s, message: %s\",\n            responseCode, urlWithParams, connection.getResponseMessage()),\n            responseCode);\n      }\n    } catch (AuthenticationException | URISyntaxException e) {\n      throw new IOException(e);\n    } finally {\n      if (connection != null) {\n        connection.disconnect();\n      }\n    }\n  }\n\n  private static void writeFileToPutRequest(Configuration conf,\n      HttpURLConnection connection, File imageFile, Canceler canceler,\n      int bufferSize)\n      throws IOException {\n    connection.setRequestProperty(Util.CONTENT_TYPE, \"application/octet-stream\");","sourceCodeStart":300,"sourceCodeEnd":336,"githubUrl":"https://github.com/apache/hadoop/blob/2add9630210752f88ceb1bb74eb65e37bf41da8e/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/TransferFsImage.java#L300-L336","documentation":"TransferFsImage.uploadImage() streams the image with HTTP PUT and checks connection.getResponseCode(); any non-200 answer raises HttpPutFailedException carrying the status code, the full URL, and the server's response message. The actionable cause is in the peer's status/message and the peer NameNode's log.","triggerScenarios":"PUT to the peer's ImageServlet (putImage endpoint) returns e.g. 401/403 on Kerberos/SPNEGO failure, 403/409 when the peer rejects the txid or another transfer is in progress, 500 on peer-side disk/IO failure, or a proxy/timeout returns 4xx/5xx because of a wrong http-address or http-policy mismatch.","commonSituations":"Expired or missing SPNEGO credentials between 2NN and NN; peer's checkpoint volume full; concurrent putImage from two secondaries; SSL/http vs https misconfiguration of dfs.namenode.http(s)-address; an intermediate proxy stripping PUT or auth headers.","solutions":["Read the embedded status and message, then check the peer NameNode log at the same timestamp for the server-side reason","For 401/403: fix Kerberos (kinit/keytab, SPNEGO principal, HTTP policy) between the two nodes","For 5xx: check the peer's checkpoint directory space and writability","Ensure a single checkpointer per namespace and correct http-address/http-policy in config, then let the next checkpoint retry"],"exampleFix":null,"handlingStrategy":"retry","validationCode":null,"typeGuard":null,"tryCatchPattern":"int attempts = 0;\nwhile (true) {\n  try {\n    TransferFsImage.uploadImage(url, conf, storage, nnf, txId, canceler);\n    break;\n  } catch (TransferFsImage.HttpPutFailedException e) {\n    int code = e.getStatusCode();\n    if (code == 401 || code == 403) throw e;       // auth: fix Kerberos, do not retry\n    if (++attempts >= 3) throw e;                   // give up after 3\n    Thread.sleep(backoffMs << attempts);            // 5xx/timeout: retry with backoff\n  }\n}","preventionTips":["Keep SPNEGO/Kerberos principals and keytabs valid on both nodes; monitor ticket expiry","Ensure dfs.namenode.http-address / http-policy match between peers (no stray proxy in between)","Watch the receiving NN's log when an upload fails — its message is the ground truth"],"tags":["hdfs","namenode","transferfsimage","http","image-upload","kerberos","checkpoint"],"backgroundTag":"http-request-failed","analyzedSha":"2add9630210752f88ceb1bb74eb65e37bf41da8e","analyzedAt":"2026-08-22T19:55:07.957Z","schemaVersion":2},"datasetVersion":"2026-08-22T20:17:22.307Z"}