apache/hadoop · critical · YarnRuntimeException

Resource Manager doesn't recognize AttemptId: {}

Error message

Resource Manager doesn't recognize AttemptId: {}

What it means

LocalContainerAllocator (MR AM running in local-allocation mode) calls scheduler.allocate and receives ApplicationAttemptNotFoundException: the ResourceManager no longer knows this attempt. Typically the RM restarted (or the attempt was evicted/forgotten), so the AM emits a JobEvent JOB_AM_REBOOT telling the job to clean up, then throws YarnRuntimeException wrapping the original cause.

Source

Thrown at hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app/src/main/java/org/apache/hadoop/mapreduce/v2/app/local/LocalContainerAllocator.java:122

  @SuppressWarnings("unchecked")
  @Override
  protected synchronized void heartbeat() throws Exception {
    AllocateRequest allocateRequest =
        AllocateRequest.newInstance(this.lastResponseID,
          super.getApplicationProgress(), new ArrayList<ResourceRequest>(),
        new ArrayList<ContainerId>(), null);
    AllocateResponse allocateResponse = null;
    try {
      allocateResponse = scheduler.allocate(allocateRequest);
      // Reset retry count if no exception occurred.
      retrystartTime = System.currentTimeMillis();
    } catch (ApplicationAttemptNotFoundException e) {
      LOG.info("Event from RM: shutting down Application Master");
      // This can happen if the RM has been restarted. If it is in that state,
      // this application must clean itself up.
      eventHandler.handle(new JobEvent(this.getJob().getID(),
        JobEventType.JOB_AM_REBOOT));
      throw new YarnRuntimeException(
        "Resource Manager doesn't recognize AttemptId: "
            + this.getContext().getApplicationID(), e);
    } catch (ApplicationMasterNotRegisteredException e) {
      LOG.info("ApplicationMaster is out of sync with ResourceManager,"
          + " hence resync and send outstanding requests.");
      this.lastResponseID = 0;
      register();
    } catch (Exception e) {
      // This can happen when the connection to the RM has gone down. Keep
      // re-trying until the retryInterval has expired.
      if (System.currentTimeMillis() - retrystartTime >= retryInterval) {
        LOG.error("Could not contact RM after " + retryInterval + " milliseconds.");
        eventHandler.handle(new JobEvent(this.getJob().getID(),
                                         JobEventType.INTERNAL_ERROR));
        throw new YarnRuntimeException("Could not contact RM after " +
                                retryInterval + " milliseconds.");
      }
      // Throw this up to the caller, which may decide to ignore it and

View on GitHub (pinned to 2add963021)

Solutions

1. Resubmit the job — after JOB_AM_REBOOT the AM is designed to shut down and the job must rerun (client retry frameworks should treat this as a fresh submission)
2. Enable work-preserving RM recovery (yarn.resourcemanager.work-preserving-recovery.enabled=true, recovery store configured) so attempts survive RM restarts
3. In test harnesses, restart RMs with state recovery configured, or stop jobs before RM restarts

try {
  job.waitForCompletion(false);
} catch (YarnRuntimeException e) {
  if (e.getMessage().contains("doesn't recognize AttemptId")) {
    // RM restarted and forgot the attempt: clean shutdown by design, resubmit the job
    resubmit(jobConf);
  } else { throw e; }
}

When it happens

Trigger: RM restart without (or with failed) recovery state while the job runs in local mode; RM HA failover dropping attempt state; the application attempt was killed at the RM but the AM process is still alive and heartbeating.

Common situations: MiniYarnCluster/MiniDFSCluster integration tests restarting the RM; single-node RM crash during local-mode debugging; work-preserving RM recovery disabled in small clusters.

Related errors

• Resource Manager doesn't recognize AttemptId: {}
• Could not contact RM after {} milliseconds.
• Could not contact RM after {} milliseconds.
• "Mkdirs failed to create " + reduceIn.getParent().toString()
• "Couldn't rename " + mapOut

AI-assisted analysis of apache/hadoop@2add963021 (2026-08-22). Data as JSON: /api/errors/e0fb49183b072ec4. Report an issue: GitHub.