{"record":{"id":"e0fb49183b072ec4","repo":"apache/hadoop","slug":"resource-manager-doesn-t-recognize-attemptid","errorCode":null,"errorMessage":"Resource Manager doesn't recognize AttemptId: {}","messagePattern":"Resource Manager doesn't recognize AttemptId: (.+?)","errorType":"exception","errorClass":"YarnRuntimeException","httpStatus":null,"severity":"critical","filePath":"hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app/src/main/java/org/apache/hadoop/mapreduce/v2/app/local/LocalContainerAllocator.java","lineNumber":122,"sourceCode":"  @SuppressWarnings(\"unchecked\")\n  @Override\n  protected synchronized void heartbeat() throws Exception {\n    AllocateRequest allocateRequest =\n        AllocateRequest.newInstance(this.lastResponseID,\n          super.getApplicationProgress(), new ArrayList<ResourceRequest>(),\n        new ArrayList<ContainerId>(), null);\n    AllocateResponse allocateResponse = null;\n    try {\n      allocateResponse = scheduler.allocate(allocateRequest);\n      // Reset retry count if no exception occurred.\n      retrystartTime = System.currentTimeMillis();\n    } catch (ApplicationAttemptNotFoundException e) {\n      LOG.info(\"Event from RM: shutting down Application Master\");\n      // This can happen if the RM has been restarted. If it is in that state,\n      // this application must clean itself up.\n      eventHandler.handle(new JobEvent(this.getJob().getID(),\n        JobEventType.JOB_AM_REBOOT));\n      throw new YarnRuntimeException(\n        \"Resource Manager doesn't recognize AttemptId: \"\n            + this.getContext().getApplicationID(), e);\n    } catch (ApplicationMasterNotRegisteredException e) {\n      LOG.info(\"ApplicationMaster is out of sync with ResourceManager,\"\n          + \" hence resync and send outstanding requests.\");\n      this.lastResponseID = 0;\n      register();\n    } catch (Exception e) {\n      // This can happen when the connection to the RM has gone down. Keep\n      // re-trying until the retryInterval has expired.\n      if (System.currentTimeMillis() - retrystartTime >= retryInterval) {\n        LOG.error(\"Could not contact RM after \" + retryInterval + \" milliseconds.\");\n        eventHandler.handle(new JobEvent(this.getJob().getID(),\n                                         JobEventType.INTERNAL_ERROR));\n        throw new YarnRuntimeException(\"Could not contact RM after \" +\n                                retryInterval + \" milliseconds.\");\n      }\n      // Throw this up to the caller, which may decide to ignore it and","sourceCodeStart":104,"sourceCodeEnd":140,"githubUrl":"https://github.com/apache/hadoop/blob/2add9630210752f88ceb1bb74eb65e37bf41da8e/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app/src/main/java/org/apache/hadoop/mapreduce/v2/app/local/LocalContainerAllocator.java#L104-L140","documentation":"LocalContainerAllocator (MR AM running in local-allocation mode) calls scheduler.allocate and receives ApplicationAttemptNotFoundException: the ResourceManager no longer knows this attempt. Typically the RM restarted (or the attempt was evicted/forgotten), so the AM emits a JobEvent JOB_AM_REBOOT telling the job to clean up, then throws YarnRuntimeException wrapping the original cause.","triggerScenarios":"RM restart without (or with failed) recovery state while the job runs in local mode; RM HA failover dropping attempt state; the application attempt was killed at the RM but the AM process is still alive and heartbeating.","commonSituations":"MiniYarnCluster/MiniDFSCluster integration tests restarting the RM; single-node RM crash during local-mode debugging; work-preserving RM recovery disabled in small clusters.","solutions":["Resubmit the job — after JOB_AM_REBOOT the AM is designed to shut down and the job must rerun (client retry frameworks should treat this as a fresh submission)","Enable work-preserving RM recovery (yarn.resourcemanager.work-preserving-recovery.enabled=true, recovery store configured) so attempts survive RM restarts","In test harnesses, restart RMs with state recovery configured, or stop jobs before RM restarts"],"exampleFix":null,"handlingStrategy":"retry","validationCode":null,"typeGuard":null,"tryCatchPattern":"try {\n  job.waitForCompletion(false);\n} catch (YarnRuntimeException e) {\n  if (e.getMessage().contains(\"doesn't recognize AttemptId\")) {\n    // RM restarted and forgot the attempt: clean shutdown by design, resubmit the job\n    resubmit(jobConf);\n  } else { throw e; }\n}","preventionTips":["Run RM with recovery enabled and a state store (ZK/leveldb) plus work-preserving restart","In tests, avoid restarting MiniYarnCluster RMs mid-job unless recovery is configured","Wrap job submission in an idempotent retry layer that handles AM reboot after RM restarts"],"tags":["mapreduce","yarn","resource-manager","rm-restart","am-reboot"],"backgroundTag":"application-attempt-not-found","analyzedSha":"2add9630210752f88ceb1bb74eb65e37bf41da8e","analyzedAt":"2026-08-22T19:55:07.957Z","schemaVersion":2},"datasetVersion":"2026-08-23T01:17:44.959Z"}