apache/hadoop · error · StateStoreUnavailableException

Cached State Store not initialized, {recordClass} records no

Error message

Cached State Store not initialized, {recordClass} records not valid

What it means

CachedRecordStore backs the Router's cached views of State Store data (membership, mount table, etc.). checkCacheAvailable() runs before serving any cached records and throws StateStoreUnavailableException unless the underlying driver isDriverReady() AND this.initialized is true — meaning the cache was never loaded for this record type or the driver went away. The message names the record class whose records cannot be trusted.

Source

Thrown at hadoop-hdfs-project/hadoop-hdfs-rbf/src/main/java/org/apache/hadoop/hdfs/server/federation/store/CachedRecordStore.java:104

   * @param clazz Class of the record to store.
   * @param driver State Store driver.
   * @param over If the entries should be overridden if they expire
   */
  protected CachedRecordStore(
      Class<R> clazz, StateStoreDriver driver, boolean over) {
    super(clazz, driver);

    this.override = over;
  }

  /**
   * Check that the cache of the State Store information is available.
   *
   * @throws StateStoreUnavailableException If the cache is not initialized.
   */
  private void checkCacheAvailable() throws StateStoreUnavailableException {
    if (!getDriver().isDriverReady() || !this.initialized) {
      throw new StateStoreUnavailableException(
          "Cached State Store not initialized, " +
          getRecordClass().getSimpleName() + " records not valid");
    }
  }

  @Override
  public boolean loadCache(boolean force) throws IOException {
    // Prevent loading the cache too frequently
    if (force || isUpdateTime()) {
      List<R> newRecords = null;
      long t = -1;
      long startTime = Time.monotonicNow();
      try {
        QueryResult<R> result = getDriver().get(getRecordClass());
        newRecords = result.getRecords();
        t = result.getTimestamp();

        // If we have any expired record, update the State Store

View on GitHub (pinned to 2add963021)

Solutions

1. Wait for Router initialization to complete and take the router out of safemode ('hdfs dfsrouteradmin -safemode leave') once the State Store is reachable.
2. Fix State Store backend connectivity (ZooKeeper/MySQL/state directory) — check the driver-specific errors in the log.
3. Confirm the State Store service is running via router JMX/state and that cache refreshes succeed.
4. Retry the request after recovery; the cache loads on the next refresh cycle and the exception stops.

// Check cache/driver readiness before issuing router RPC-dependent work
if (!stateStore.isDriverReady()
    || !stateStore.getRecordStore(MembershipState.class).isInitialized()) {
  // wait for the state store cache load instead of failing the request
}

boolean isStateStoreUnavailable(Throwable t) {
  return t instanceof StateStoreUnavailableException
      || (t instanceof IOException && t.getMessage() != null
          && t.getMessage().contains("Cached State Store not initialized"));
}

try {
  return routerRpcServer.getFileInfo(path);
} catch (StateStoreUnavailableException e) {
  // router cache not loaded yet (startup race or store outage):
  // back off and retry — the cache loads on the next refresh cycle
  Thread.sleep(1000L * attempt);
}

When it happens

Trigger: An RPC or mount-table resolution hits the cached store before the State Store service completed its first loadCache (router startup race); the state store driver transitioned to not-ready (ZK/MySQL/file backend down) while the cache is marked uninitialized; router still in startup/safemode state serving requests.

Common situations: Router just restarted and clients or a load balancer immediately send requests; state store backend outage; cache refresh failing repeatedly so the store never becomes initialized.

Related errors

• Federated metrics is not initialized
• Cannot serialize field {} into JSON
• Namenode metrics is not initialized
• Mount table state store is not available.
• Disabled Nameservice state store is not available.

AI-assisted analysis of apache/hadoop@2add963021 (2026-08-22). Data as JSON: /api/errors/90488c44df7ab16e. Report an issue: GitHub.