{"record":{"id":"bf84fa95584348d8","repo":"nathanmarz/storm","slug":"a-single-worker-should-have-1-systembolt-instance","errorCode":null,"errorMessage":"A single worker should have 1 SystemBolt instance.","messagePattern":"A single worker should have 1 SystemBolt instance\\.","errorType":"exception","errorClass":"RuntimeException","httpStatus":null,"severity":"error","filePath":"storm-core/src/jvm/backtype/storm/metric/SystemBolt.java","lineNumber":95,"sourceCode":"            Long collectionTimeP = _gcBean.getCollectionTime();\n\n            Map ret = null;\n            if(_collectionCount!=null && _collectionTime!=null) {\n                ret = new HashMap();\n                ret.put(\"count\", collectionCountP - _collectionCount);\n                ret.put(\"timeMs\", collectionTimeP - _collectionTime);\n            }\n\n            _collectionCount = collectionCountP;\n            _collectionTime = collectionTimeP;\n            return ret;\n        }\n    }\n\n    @Override\n    public void prepare(final Map stormConf, TopologyContext context, OutputCollector collector) {\n        if(_prepareWasCalled && !\"local\".equals(stormConf.get(Config.STORM_CLUSTER_MODE))) {\n            throw new RuntimeException(\"A single worker should have 1 SystemBolt instance.\");\n        }\n        _prepareWasCalled = true;\n\n        int bucketSize = RT.intCast(stormConf.get(Config.TOPOLOGY_BUILTIN_METRICS_BUCKET_SIZE_SECS));\n\n        final RuntimeMXBean jvmRT = ManagementFactory.getRuntimeMXBean();\n\n        context.registerMetric(\"uptimeSecs\", new IMetric() {\n            @Override\n            public Object getValueAndReset() {\n                return jvmRT.getUptime()/1000.0;\n            }\n        }, bucketSize);\n\n        context.registerMetric(\"startTimeSecs\", new IMetric() {\n            @Override\n            public Object getValueAndReset() {\n                return jvmRT.getStartTime()/1000.0;","sourceCodeStart":77,"sourceCodeEnd":113,"githubUrl":"https://github.com/nathanmarz/storm/blob/cdb116e942666973bc4eaa0df098d5bab82739e7/storm-core/src/jvm/backtype/storm/metric/SystemBolt.java#L77-L113","documentation":"SystemBolt is a built-in bolt that should exist exactly once per worker; a static flag _prepareWasCalled detects a second prepare() call in distributed mode and throws. This usually means the worker spawned more than one SystemBolt instance — typically because the topology or its config registers the __system bolt multiple times (e.g. duplicate builtin metric registrations) rather than user code calling prepare directly.","triggerScenarios":"In distributed (non-'local') cluster mode, SystemBolt.prepare is invoked a second time within the same worker JVM, i.e. two SystemBolt instances are created for one worker (duplicate __system component registration or a Storm/config issue causing double instantiation).","commonSituations":"Running on a real cluster (not LocalCluster) after customizing topology metrics consumer/builtin metric settings that reference __system twice; topology definitions built programmatically with duplicated bolt declarations; Storm version upgrades where worker topology-loading logic changed.","solutions":["Inspect the topology definition and remove duplicate declarations of the __system bolt (it should appear once per topology).","Check Config.TOPOLOGY_BUILTIN_METRICS_BUCKET_SIZE_SECS and metrics registration code for logic that adds __system more than once.","If using a wrapper/test harness, ensure it does not reuse the same SystemBolt class across topologies in one worker; in local mode this check is skipped by design.","Compare against the stock default topology config; restore the default builtin-metrics registration if it was customized."],"exampleFix":"// before\ntopologyBuilder.setBolt(\"__system\", new SystemBolt(), 1);\ntopologyBuilder.setBolt(\"__system\", new SystemBolt(), 1); // duplicate component name/instance\n// after\ntopologyBuilder.setBolt(\"__system\", new SystemBolt(), 1); // only once per topology","handlingStrategy":"validation","validationCode":"// Before submitting, ensure __system is declared at most once in the topology\nlong sysCount = topology.getBolts().keySet().stream()\n    .filter(n -> n.equals(\"__system\")).count();\nif (sysCount > 1) throw new IllegalStateException(\"__system bolt declared \" + sysCount + \" times\");","typeGuard":null,"tryCatchPattern":"// Wrap topology submission on a real cluster\ntry {\n    stormSubmitter.submitTopology(name, conf, topology);\n} catch (RuntimeException e) {\n    if (e.getMessage() != null && e.getMessage().contains(\"1 SystemBolt instance\")) {\n        // topology registers __system more than once; fix topology definition and resubmit\n    } else throw e;\n}","preventionTips":["Never declare the __system bolt yourself unless you know exactly why; let Storm's defaults handle it.","Avoid programmatic loops that add builtin bolts; dedupe component names.","Test new configs first in LocalCluster (the guard is skipped there) then on a small cluster.","Track builtin-bolt related changes when upgrading Storm versions."],"tags":["metrics","topology","cluster-mode","storm"],"backgroundTag":"internal-invariant-violation","analyzedSha":"cdb116e942666973bc4eaa0df098d5bab82739e7","analyzedAt":"2026-09-12T14:30:00.714Z","contentChangedAt":"2026-09-12T14:30:00.714Z","schemaVersion":2},"datasetVersion":"2026-09-16T09:17:16.951Z"}