code4craft/webmagic · error · IllegalStateException

Already closed!

Error message

Already closed!

What it means

WebDriverPool.checkRunning() verifies the pool's atomic state is still STAT_RUNNING via compareAndSet(STAT_RUNNING, STAT_RUNNING). If the pool has been closed (closeAll() or shutdown) or was never properly running, it throws IllegalStateException "Already closed!". get() and returnToPool() both call this, so any pool usage after closing fails.

Solutions

1. Don't call get()/returnToPool() after closeAll(); create a new WebDriverPool instance if drivers are needed again (the pool cannot be reopened).
2. Move closeAll() to the application's final shutdown hook only, not per-request code paths.
3. Guard usage with a lifecycle flag or synchronize pool access so close and borrow don't race.
4. Check for double-close in finally blocks (e.g. closeAll called in both a request handler and a shutdown hook).

Example fix

// before
try {
    driver = pool.get(url);
    ...
} finally {
    pool.closeAll(); // closes pool for everyone
}
// after
driver = pool.get(url);
...
pool.returnToPool(driver); // closeAll() only at application shutdown

// pool state is internal; track lifecycle yourself
if (poolClosed) throw new IllegalStateException("Pool already closed; create a new WebDriverPool");

try {
    WebDriver driver = pool.get(url);
} catch (IllegalStateException e) {
    if ("Already closed!".equals(e.getMessage())) {
        throw new IllegalStateException("WebDriverPool was closed; rebuild it or fix your shutdown ordering", e);
    }
    throw e;
}

When it happens

Trigger: Calling pool.get(...) or pool.returnToPool(driver) after pool.closeAll() (or quitAll/shutdown) has flipped the state to STAT_CLODED; double-closing and then reusing the pool; concurrent close while another thread borrows a driver.

Common situations: App shutdown hooks closing the pool while scheduled crawling tasks still run; accidentally calling closeAll() in a finally block per-request instead of at application end; sharing one pool instance across threads that race close vs. get.

Understand the failure class

Background: "Invalid state transition" errors: "status must be X, actually Y", "already rejected/charging/uninstalled", "cannot ... while running" — what they mean when a library rejects your call — this error's family across 31 libraries.

Related errors

• Spider is already running!
• Property ' ' not set!
• init cache scheduler error
• wrong proto type map
• XPath error!

AI-assisted analysis of code4craft/webmagic@67816a19d6 (2026-09-08). Data as JSON: /api/errors/f2ee4730fdb5a50b. Report an issue: GitHub.

Appendix: source

Thrown at webmagic-selenium/src/main/java/us/codecraft/webmagic/downloader/selenium/WebDriverPool.java:223

					// ChromeDriver e = new ChromeDriver();
					// WebDriver e = getWebDriver();
					// innerQueue.add(e);
					// webDriverList.add(e);
				}
			}

		}
		return innerQueue.take();
	}

	public void returnToPool(WebDriver webDriver) {
		checkRunning();
		innerQueue.add(webDriver);
	}

	protected void checkRunning() {
		if (!stat.compareAndSet(STAT_RUNNING, STAT_RUNNING)) {
			throw new IllegalStateException("Already closed!");
		}
	}

	public void closeAll() {
		boolean b = stat.compareAndSet(STAT_RUNNING, STAT_CLODED);
		if (!b) {
			throw new IllegalStateException("Already closed!");
		}
		for (WebDriver webDriver : webDriverList) {
			logger.info("Quit webDriver" + webDriver);
			webDriver.quit();
			webDriver = null;
		}
	}

}

View on GitHub (pinned to 67816a19d6)