apache/hadoop · error · YarnRuntimeException

Could not connect to History server.

Error message

Could not connect to History server.

What it means

Thrown by the MapReduce job client's ClientCache.getClient when instantiateHistoryProxy() fails with IOException — the client could not create an RPC proxy to the MapReduce JobHistoryServer. The IOException is wrapped in a YarnRuntimeException with this message, and both are logged at WARN. This is a client-side setup failure, not a job failure: every status poll after job completion goes through the history proxy, so this breaks job status/log retrieval.

Source

Thrown at hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-jobclient/src/main/java/org/apache/hadoop/mapred/ClientCache.java:63

  private Map<JobID, ClientServiceDelegate> cache = 
      new HashMap<JobID, ClientServiceDelegate>();

  private MRClientProtocol hsProxy;

  public ClientCache(Configuration conf, ResourceMgrDelegate rm) {
    this.conf = conf;
    this.rm = rm;
  }

  //TODO: evict from the cache on some threshold
  public synchronized ClientServiceDelegate getClient(JobID jobId) {
    if (hsProxy == null) {
      try {
        hsProxy = instantiateHistoryProxy();
      } catch (IOException e) {
        LOG.warn("Could not connect to History server.", e);
        throw new YarnRuntimeException("Could not connect to History server.", e);
      }
    }
    ClientServiceDelegate client = cache.get(jobId);
    if (client == null) {
      client = new ClientServiceDelegate(conf, rm, jobId, hsProxy);
      cache.put(jobId, client);
    }
    return client;
  }

  protected synchronized MRClientProtocol getInitializedHSProxy()
      throws IOException {
    if (this.hsProxy == null) {
      hsProxy = instantiateHistoryProxy();
    }
    return this.hsProxy;
  }
  

View on GitHub (pinned to 2add963021)

Solutions

1. Verify the JobHistoryServer is running: check the process and http://<jhs>:19888/ws/v1/history/info
2. Set mapreduce.jobhistory.address to the JHS's routable hostname:10020 in mapred-site.xml and restart the client
3. Check firewall/network path to port 10020 from the client node
4. In Kerberos clusters confirm mapreduce.jobhistory.principal and keytab are correct so proxy creation succeeds

Example fix

<!-- before -->
<property><name>mapreduce.jobhistory.address</name><value>0.0.0.0:10020</value></property>

<!-- after -->
<property><name>mapreduce.jobhistory.address</name><value>jhs.example.com:10020</value></property>
<!-- ensure: mr-jobhistory-daemon.sh start historyserver -->

Configuration conf = new Configuration();
String jhsAddr = conf.get("mapreduce.jobhistory.address", "0.0.0.0:10020");
String host = jhsAddr.split(":")[0];
if ("0.0.0.0".equals(host)) throw new IllegalStateException(
  "mapreduce.jobhistory.address must be a routable hostname");
// optionally TCP-probe host:10020 before submitting jobs

try {
  client = clientCache.getClient(jobId);
} catch (YarnRuntimeException e) {
  if (e.getMessage().contains("Could not connect to History server")) {
    // fix env: start JHS / correct mapreduce.jobhistory.address, then retry
    ensureJhsReachable(); client = clientCache.getClient(jobId);
  } else throw e;
}

When it happens

Trigger: Running a MapReduce job on YARN where mapreduce.jobhistory.address points to a host/port with no running JobHistoryServer, or the JHS RPC port is firewalled/unreachable. Also when yarn.app.mapreduce.am.jobhistory... recovery data forces the client to consult history and the proxy creation (YarnRPC call) throws.

Common situations: JHS not started (most common in small/pseudo-distributed clusters); mapreduce.jobhistory.address left as 0.0.0.0:10020 in mapred-site.xml — 0.0.0.0 is not routable from remote clients; JHS bound to a different address than the one configured, or started after DNS change; Kerberos: missing jobhistory principal/keytab causing proxy setup IOException

Related errors

• Error in instantiating YarnClient
• Unrecognized task type: {}
• Unrecognized State: {}
• Unrecognized Phase: {}
• Unrecognized status: {}

AI-assisted analysis of apache/hadoop@2add963021 (2026-08-22). Data as JSON: /api/errors/58aca37bb279d53f. Report an issue: GitHub.