{"record":{"id":"e8a9a44f7a13c290","repo":"apache/pulsar","slug":"failed-to-open-state-table-for-function-tenant-e8a9a4","errorCode":null,"errorMessage":"Failed to open state table for function ${tenant}/${namespace}/${name}","messagePattern":"Failed to open state table for function (.+?)/(.+?)/(.+?)","errorType":"exception","errorClass":"IOException","httpStatus":null,"severity":"critical","filePath":"pulsar-functions/instance/src/main/java/org/apache/pulsar/functions/instance/state/BKStateStoreProviderImpl.java","lineNumber":199,"sourceCode":"        while (openSw.elapsed(TimeUnit.MINUTES) < 1) {\n            try {\n                return result(client.openTable(name), 1, TimeUnit.MINUTES);\n            } catch (InternalServerException ise) {\n                log.warn()\n                        .attr(\"tenant\", tenant)\n                        .attr(\"namespace\", namespace)\n                        .attr(\"name\", name)\n                        .attr(\"message\", ise.getMessage())\n                        .log(\"Encountered internal server on opening state table,\"\n                                + \" re-attempt in 100 milliseconds\");\n                TimeUnit.MILLISECONDS.sleep(100);\n            } catch (TimeoutException e) {\n                throw new RuntimeException(\n                        \"Failed to open state table for function \" + tenant + \"/\" + namespace + \"/\" + name\n                                + \" within timeout period\", e);\n            }\n        }\n        throw new IOException(\"Failed to open state table for function \" + tenant + \"/\" + namespace + \"/\" + name);\n    }\n\n    @SuppressWarnings(\"unchecked\")\n    @Override\n    public <T extends StateStore> T getStateStore(String tenant, String namespace, String name) throws Exception {\n        // we defer creation of the state table until a java instance is running here.\n        createStateTable(stateStorageServiceUrl, tenant, namespace, name);\n        Table<ByteBuf, ByteBuf> table = openStateTable(tenant, namespace, name);\n        return (T) new BKStateStoreImpl(tenant, namespace, name, table);\n    }\n\n    @Override\n    public void cleanUp(String tenant, String namespace, String name) throws Exception {\n        StorageAdminClient storageAdminClient = new SimpleStorageAdminClientImpl(\n                StorageClientSettings.newBuilder().serviceUri(stateStorageServiceUrl).build(),\n                ClientResources.create().scheduler());\n        String tableNs = FunctionCommon.getStateNamespace(tenant, namespace);\n        storageAdminClient.deleteStream(tableNs, name).whenComplete((res, throwable) -> {","sourceCodeStart":181,"sourceCodeEnd":217,"githubUrl":"https://github.com/apache/pulsar/blob/820761864ed8e2a7d2e52dd9763ad2ae117c1395/pulsar-functions/instance/src/main/java/org/apache/pulsar/functions/instance/state/BKStateStoreProviderImpl.java#L181-L217","documentation":"Thrown by BKStateStoreProviderImpl.openStateTable when the BookKeeper-backed state table for a function cannot be opened. Two variants: a timeout wrapped in a RuntimeException, and a generic IOException when opening fails for any other reason. The state table is the keyed-state storage used by functions for stateful processing (e.g. windowing, count state).","triggerScenarios":"Calling openStateTable (via the `table` lambda) when the BookKeeper table service cannot create/open the state table within the timeout (TimeoutException), or fails outright (any other failure producing IOException).","commonSituations":"BookKeeper cluster unreachable or overloaded; metadata store (ZooKeeper) connectivity problems; the state table already exists with an incompatible format; slow disk/network causing creation to exceed the timeout.","solutions":["Verify the BookKeeper service is reachable from the function worker and healthy (bookies up, enough writable bookies).","Check metadata store (ZooKeeper) connectivity and configuration of the state storage service.","Increase the state table open timeout or investigate why table creation is slow (disk pressure, network latency).","Inspect worker logs for the underlying cause; if a previous partial creation corrupted the table, delete the stale state table and restart the function."],"exampleFix":"// before: function fails at startup with IOException\n// after: ensure state storage configured and bookies healthy\n//   broker.conf: stateStorageServiceImpl=org.apache.pulsar.functions.instance.state.PulsarMetadataStoreStateStoreProviderImpl\n//   verify: pulsar-admin bookies list-bookies","handlingStrategy":"retry","validationCode":"// check bookies and metadata store before starting the function\npulsar-admin bookies list-bookies\npulsar-admin brokers get-runtime-config | grep stateStorage","typeGuard":null,"tryCatchPattern":"try {\n  stateStore = provider.openStateTable(tenant, ns, name);\n} catch (RuntimeException | IOException e) {\n  log.error(\"state table open failed\", e);\n  // backoff and retry, then fail the function\n}","preventionTips":["Monitor BookKeeper bookie availability before deploying stateful functions","Keep metadata store reachable from function workers","Alert on state table open latency"],"tags":["bookkeeper","state-storage","timeout","pulsar-functions"],"backgroundTag":"state-table-open-failed","analyzedSha":"820761864ed8e2a7d2e52dd9763ad2ae117c1395","analyzedAt":"2026-09-06T00:14:20.138Z","contentChangedAt":"2026-09-06T00:14:20.138Z","schemaVersion":2},"datasetVersion":"2026-09-14T05:17:10.506Z"}