{"record":{"id":"fffacbda4c2f3b0e","repo":"apache/hadoop","slug":"pathnotfound-fffacb","errorCode":"PathNotFound","errorMessage":"Path had to be recovered from atomic rename operation.","messagePattern":"Path had to be recovered from atomic rename operation\\.","errorType":"http","errorClass":"AbfsRestOperationException","httpStatus":404,"severity":"error","filePath":"hadoop-tools/hadoop-azure/src/main/java/org/apache/hadoop/fs/azurebfs/services/AbfsBlobClient.java","lineNumber":1877,"sourceCode":"      renameAtomicity.redo();\n      renameSrcHasChanged = false;\n    } catch (AbfsRestOperationException ex) {\n      /*\n       * At this point, the source marked by the renamePending json file, might have\n       * already got renamed by some parallel thread, or at this point, the path\n       * would have got modified which would result in eTag change, which would lead\n       * to a HTTP_CONFLICT. In this case, no more operation needs to be taken, and\n       * the calling getPathStatus can return this source path as result.\n       */\n      if (ex.getStatusCode() == HTTP_NOT_FOUND\n          || ex.getStatusCode() == HTTP_CONFLICT) {\n        renameSrcHasChanged = true;\n      } else {\n        throw ex;\n      }\n    }\n    if (!renameSrcHasChanged) {\n      throw new AbfsRestOperationException(\n          AzureServiceErrorCode.PATH_NOT_FOUND.getStatusCode(),\n          AzureServiceErrorCode.PATH_NOT_FOUND.getErrorCode(),\n          ATOMIC_DIR_RENAME_RECOVERY_ON_GET_PATH_EXCEPTION,\n          null);\n    }\n  }\n\n  /**\n   * Redo the rename operation when path is present in atomic directory list\n   * or when path has {@link RenameAtomicity#SUFFIX} suffix.\n   *\n   * @param path path of the pendingJson for the atomic path.\n   * @param renamePendingJsonLen length of the pendingJson file.\n   * @param tracingContext tracing context.\n   *\n   * @throws AzureBlobFileSystemException server error\n   */\n","sourceCodeStart":1859,"sourceCodeEnd":1895,"githubUrl":"https://github.com/apache/hadoop/blob/2add9630210752f88ceb1bb74eb65e37bf41da8e/hadoop-tools/hadoop-azure/src/main/java/org/apache/hadoop/fs/azurebfs/services/AbfsBlobClient.java#L1859-L1895","documentation":"Thrown during rename-atomicity recovery on flat-namespace (non-HNS) Azure Storage accounts. After an interrupted rename, AbfsBlobClient re-checks the source path with GetPathStatus; only HTTP 404 or HTTP 409 proves the source disappeared or changed. If that status call fails with any other error, the client cannot verify the rename and surfaces PATH_NOT_FOUND with the recovery message.","triggerScenarios":"A rename on a non-HNS account goes through the atomic-rename recovery flow (rename-pending JSON / RenameAtomicity SUFFIX handling), and the follow-up GetPathStatus on the source throws an AbfsRestOperationException whose status is neither 404 nor 409 (e.g. 500, 403, timeout), so renameSrcHasChanged stays false and this synthetic PATH_NOT_FOUND is thrown.","commonSituations":"Transient server errors or throttling during rename recovery on the blob endpoint; a proxy or middlebox returning unexpected status codes; races where another client concurrently finishes or rolls back the same rename; leftover rename-pending entries after a crashed job that is retried.","solutions":["Retry the file-system operation (rename/open) with backoff — once the transient GetPathStatus error clears, the recovery usually completes.","Check for concurrent writers finishing the same rename; verify the destination now exists before retrying.","If a job crashed mid-rename, inspect the container for leftover rename-pending JSON / atomic-rename suffix files and clean them once no writer is active.","If persistent, capture the underlying status code and request-id from logs and investigate account-side issues (throttling, firewall, soft-delete)."],"exampleFix":"// before\nfs.rename(src, dst); // fails once during transient outage on non-HNS recovery\n// after\nboolean ok = false;\nfor (int i = 0; i < 3 && !ok; i++) {\n  try { ok = fs.rename(src, dst); }\n  catch (AbfsRestOperationException ex) {\n    if (ex.getStatusCode() != 404) { throw ex; }\n    sleepBackoff(i);\n  }\n}","handlingStrategy":"retry","validationCode":null,"typeGuard":null,"tryCatchPattern":"catch (AbfsRestOperationException ex) {\n  if (ex.getStatusCode() == 404\n      && ex.getMessage().contains(\"atomic rename\")) {\n    // verify destination exists, then retry the rename with backoff\n  } else {\n    throw ex;\n  }\n}","preventionTips":["Prefer HNS-enabled accounts for rename-heavy pipelines; the non-HNS atomic-rename recovery path is what throws this.","Never run two clients renaming the same source path concurrently.","Make retry logic re-check the destination before re-issuing a failed rename."],"tags":["azure","abfs","rename","non-hns","recovery","path-not-found"],"backgroundTag":"atomic-rename-recovery","analyzedSha":"2add9630210752f88ceb1bb74eb65e37bf41da8e","analyzedAt":"2026-08-22T19:55:07.957Z","schemaVersion":2},"datasetVersion":"2026-08-22T20:17:22.307Z"}