{"record":{"id":"c3e430aabe87a58d","repo":"apache/iceberg","slug":"failed-to-start-an-embedded-metastore-because-embe","errorCode":null,"errorMessage":"Failed to start an embedded metastore because embedded Derby supports only one client at a time. To fix this, use a metastore that supports multiple clients.","messagePattern":"Failed to start an embedded metastore because embedded Derby supports only one client at a time\\. To fix this, use a metastore that supports multiple clients\\.","errorType":"exception","errorClass":"RuntimeMetaException","httpStatus":null,"severity":"error","filePath":"hive-metastore/src/main/java/org/apache/iceberg/hive/HiveClientPool.java","lineNumber":78,"sourceCode":"  protected IMetaStoreClient newClient() {\n    try {\n      try {\n        return GET_CLIENT.invoke(\n            hiveConf, (HiveMetaHookLoader) tbl -> null, HiveMetaStoreClient.class.getName());\n      } catch (RuntimeException e) {\n        // any MetaException would be wrapped into RuntimeException during reflection, so let's\n        // double-check type here\n        if (e.getCause() instanceof MetaException) {\n          throw (MetaException) e.getCause();\n        }\n        throw e;\n      }\n    } catch (MetaException e) {\n      throw new RuntimeMetaException(e, \"Failed to connect to Hive Metastore\");\n    } catch (Throwable t) {\n      if (t.getMessage() != null\n          && t.getMessage().contains(\"Another instance of Derby may have already booted\")) {\n        throw new RuntimeMetaException(\n            t,\n            \"Failed to start an embedded metastore because embedded \"\n                + \"Derby supports only one client at a time. To fix this, use a metastore that supports \"\n                + \"multiple clients.\");\n      }\n\n      throw new RuntimeMetaException(t, \"Failed to connect to Hive Metastore\");\n    }\n  }\n\n  @Override\n  protected IMetaStoreClient reconnect(IMetaStoreClient client) {\n    try {\n      client.close();\n      client.reconnect();\n    } catch (MetaException e) {\n      throw new RuntimeMetaException(e, \"Failed to reconnect to Hive Metastore\");\n    }","sourceCodeStart":60,"sourceCodeEnd":96,"githubUrl":"https://github.com/apache/iceberg/blob/86d9c8fc543e7c56c9f624eb725f76c9baff9570/hive-metastore/src/main/java/org/apache/iceberg/hive/HiveClientPool.java#L60-L96","documentation":"HiveClientPool.newClient() detects the Derby message 'Another instance of Derby may have already booted' and raises a dedicated RuntimeMetaException explaining that an embedded Derby metastore allows only one client at a time. It occurs in local/dev setups where multiple clients share one embedded Derby database.","triggerScenarios":"Two or more processes (or two HiveCatalog instances in one JVM with separate client pools) opening an embedded metastore backed by the same Derby data directory; the second client's newClient() sees the Derby boot-conflict MetaException.","commonSituations":"Local testing where a unit test JVM and a Spark shell both use an embedded metastore; a previous test JVM crashed leaving the Derby lock; running multiple workers against a dev config that never pointed at a real remote metastore.","solutions":["Point hive.metastore.uris at a real (remote) metastore service instead of using an embedded one — this is the fix the message itself recommends.","Stop the other process holding the Derby lock, or remove stale Derby lck/db files from the metastore directory after confirming nothing is running.","Ensure only one embedded-metastore client runs at a time (serialize tests, use testcontainers with a dedicated metastore per worker).","Use a shared test metastore (e.g., a Dockerized Postgres-backed Hive metastore) for parallel test runs."],"exampleFix":"// before\nconf.set(\"hive.metastore.uris\", \"\"); // embedded metastore, Derby-locked\n// after\nconf.set(\"hive.metastore.uris\", \"thrift://localhost:9083\"); // run metastore service\nHiveCatalog catalog = new HiveCatalog();\ncatalog.setConf(conf);","handlingStrategy":"validation","validationCode":"// detect embedded-metastore usage before connecting\nif (conf.get(\"hive.metastore.uris\") == null || conf.get(\"hive.metastore.uris\").isEmpty()) {\n  throw new IllegalStateException(\"Embedded metastore (Derby) not safe for multi-client use; set hive.metastore.uris\");\n}","typeGuard":null,"tryCatchPattern":"try {\n  catalog.initialize(\"hive\", conf);\n} catch (RuntimeMetaException e) {\n  if (e.getMessage().contains(\"embedded Derby\")) {\n    // stop competing client or switch to remote metastore, then retry\n  }\n}","preventionTips":["Always set hive.metastore.uris to a real metastore for anything beyond a single local client","Run test suites against a shared remote/containerized metastore, not embedded Derby","Clean up stale Derby lock files after crashed local test JVMs"],"tags":["hive","derby","embedded-metastore","local-development"],"backgroundTag":"connection-refused","analyzedSha":"86d9c8fc543e7c56c9f624eb725f76c9baff9570","analyzedAt":"2026-09-12T00:46:39.097Z","contentChangedAt":"2026-09-12T00:46:39.097Z","schemaVersion":2},"datasetVersion":"2026-09-23T08:17:48.524Z"}