{"record":{"id":"542e865e29c8e501","repo":"code4craft/webmagic","slug":"spider-is-already-running","errorCode":null,"errorMessage":"Spider is already running!","messagePattern":"Spider is already running!","errorType":"exception","errorClass":"IllegalStateException","httpStatus":null,"severity":"error","filePath":"webmagic-core/src/main/java/us/codecraft/webmagic/Spider.java","lineNumber":392,"sourceCode":"            for (SpiderListener spiderListener : spiderListeners) {\n                spiderListener.onError(request, e);\n            }\n        }\n    }\n\n    protected void onSuccess(Request request) {\n        if (CollectionUtils.isNotEmpty(spiderListeners)) {\n            for (SpiderListener spiderListener : spiderListeners) {\n                spiderListener.onSuccess(request);\n            }\n        }\n    }\n\n    private void checkRunningStat() {\n        while (true) {\n            int statNow = stat.get();\n            if (statNow == STAT_RUNNING) {\n                throw new IllegalStateException(\"Spider is already running!\");\n            }\n            if (stat.compareAndSet(statNow, STAT_RUNNING)) {\n                break;\n            }\n        }\n    }\n\n    public void close() {\n        destroyEach(downloader);\n        destroyEach(pageProcessor);\n        destroyEach(scheduler);\n        for (Pipeline pipeline : pipelines) {\n            destroyEach(pipeline);\n        }\n        threadPool.shutdown();\n    }\n\n    private void destroyEach(Object object) {","sourceCodeStart":374,"sourceCodeEnd":410,"githubUrl":"https://github.com/code4craft/webmagic/blob/67816a19d68a4fec4657bf1336227e046e251df2/webmagic-core/src/main/java/us/codecraft/webmagic/Spider.java#L374-L410","documentation":"Spider.checkRunningStat() uses an AtomicInteger stat guard to ensure a Spider instance only runs once. If another thread (or an overlapping call) has already moved the spider into STAT_RUNNING, it throws IllegalStateException instead of starting a duplicate crawl.","triggerScenarios":"Calling spider.run() (or runAsync/thread pool execution) on the same Spider instance that is already executing, or on a spider whose stat was left at STAT_RUNNING after an unclean stop.","commonSituations":"Re-invoking run() after runAsync(); sharing one Spider object across multiple threads; wrapping run() in a loop that restarts without calling stop(); a previous run crashed before stat returned to STAT_STOPPED.","solutions":["Ensure only one invocation: call run() OR runAsync() once per Spider instance","Create a new Spider (e.g. spider.clone() or WebMagic spider factory) for each crawl instead of reusing the running one","Call stop() and wait for the previous run to fully finish before restarting","Check spider.getStatus() before starting a run"],"exampleFix":"// before\nspider.run(); // second invocation on same spider while running -> IllegalStateException\n// after\nif (spider.getStatus() == Status.STOPPED) {\n    spider.run();\n} else {\n    spider.clone().run();\n}","handlingStrategy":"try-catch","validationCode":"if (spider.getStatus() == Status.RUNNING) { throw ... or create a new spider }","typeGuard":null,"tryCatchPattern":"try { spider.run(); } catch (IllegalStateException e) { /* spider busy: clone or reuse existing run */ }","preventionTips":["Run each Spider instance exactly once","Prefer runAsync() and keep a single start point","Guard with getStatus() before starting","Use spider.clone() to get a fresh instance with the same config"],"tags":["concurrency","lifecycle","illegal-state"],"backgroundTag":"invalid-state-transition","analyzedSha":"67816a19d68a4fec4657bf1336227e046e251df2","analyzedAt":"2026-09-08T11:15:28.425Z","contentChangedAt":"2026-09-08T11:15:28.425Z","schemaVersion":2},"datasetVersion":"2026-09-16T04:17:20.429Z"}