apache/hadoop · error · UnsupportedOperationException

is expected to be an instance of…

Error message

{} is expected to be an instance of org.apache.hadoop.mapreduce.v2.hs.CachedHistoryStorage

What it means

refreshLoadedJobCache reloads the loaded-jobs cache but is only meaningful for CachedHistoryStorage, the default HistoryStorage. If a custom storage implementation was configured (JobHistory.createHistoryStorage reads the storage class from configuration), the instanceof check fails and UnsupportedOperationException names the mismatched class. The call is a no-op-with-warning when the service is not STARTED, so this exception specifically means: running service plus non-cached storage.

Solutions

1. Remove the custom mapreduce.jobhistory.storage.class setting to fall back to CachedHistoryStorage if cache refresh is needed.
2. Make the custom class extend org.apache.hadoop.mapreduce.v2.hs.CachedHistoryStorage.
3. Skip invoking refreshLoadedJobCache when a non-cached storage is deployed.

Example fix

// before
jobHistory.refreshLoadedJobCache();

// after
if (jobHistory.getHistoryStorage()
    instanceof org.apache.hadoop.mapreduce.v2.hs.CachedHistoryStorage) {
  jobHistory.refreshLoadedJobCache();
}

private static boolean isCachedStorage(HistoryStorage storage) {
  return storage instanceof org.apache.hadoop.mapreduce.v2.hs.CachedHistoryStorage;
}

When it happens

Trigger: Invoking the admin refresh operation (refreshLoadedJobCache) while a custom HistoryStorage class is configured; a storage wrapper that does not extend CachedHistoryStorage.

Common situations: Deployments plugging in custom history storage and then invoking cache refresh; configuration copied from another deployment that sets the storage class.

Understand the failure class

Background: Type mismatch errors: IllegalArgumentException, TypeError and type guards across 150 open-source libraries — this error's family across 150 libraries.

Related errors

• Cannot recover task output for first attempt...
• Could not load history file
• Could not parse history file
• Could not rename to
• Could not rename to

AI-assisted analysis of apache/hadoop@2add963021 (2026-08-22). Data as JSON: /api/errors/a8cc7c85474867e7. Report an issue: GitHub.

Appendix: source

Thrown at hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-hs/src/main/java/org/apache/hadoop/mapreduce/v2/hs/JobHistory.java:249

    // currently there is 1 to 1 mapping between app and job id
    org.apache.hadoop.mapreduce.JobID oldJobID = TypeConverter.fromYarn(appID);
    Map<JobId, Job> jobs = new HashMap<JobId, Job>();
    JobId jobID = TypeConverter.toYarn(oldJobID);
    jobs.put(jobID, getJob(jobID));
    return jobs;
  }

  @Override
  public Map<JobId, Job> getAllJobs() {
    return storage.getAllPartialJobs();
  }

  public void refreshLoadedJobCache() {
    if (getServiceState() == STATE.STARTED) {
      if (storage instanceof CachedHistoryStorage) {
        ((CachedHistoryStorage) storage).refreshLoadedJobCache();
      } else {
        throw new UnsupportedOperationException(storage.getClass().getName()
            + " is expected to be an instance of "
            + CachedHistoryStorage.class.getName());
      }
    } else {
      LOG.warn("Failed to execute refreshLoadedJobCache: JobHistory service is not started");
    }
  }

  @VisibleForTesting
  HistoryStorage getHistoryStorage() {
    return storage;
  }
  
  /**
   * Look for a set of partial jobs.
   * 
   * @param offset
   *          the offset into the list of jobs.

View on GitHub (pinned to 2add963021)