code4craft/webmagic · error · IllegalArgumentException

java.io.IOException

Error message

java.io.IOException

What it means

During ScriptProcessor construction, the language's define file (language.getDefineFile()) is read from the classpath into a String with IOUtils.toString. If reading the stream throws IOException (stream closed/unreadable — and notably if getResourceAsStream returned null, NPE aside, or an I/O fault occurs), it is rethrown as IllegalArgumentException whose message is just "java.io.IOException: ...".

Solutions

1. Verify language.getDefineFile() exists on the runtime classpath (e.g. check ScriptProcessor.class.getClassLoader().getResource(defineFile) != null before constructing).
2. If using a custom Language, bundle its define file in src/main/resources and rebuild the jar.
3. Check the jar/classpath packaging (maven-shade/resources filtering) to ensure .txt/.js resource files are not filtered out.
4. Catch IllegalArgumentException during construction and log the cause's cause for the underlying stream problem.

Example fix

// before
InputStream in = getClass().getClassLoader().getResourceAsStream(language.getDefineFile());
new ScriptProcessor(language, script, 1);
// after
if (getClass().getClassLoader().getResource(language.getDefineFile()) == null) {
    throw new IllegalStateException("Missing define file: " + language.getDefineFile());
}
new ScriptProcessor(language, script, 1);

String defFile = language.getDefineFile();
if (ScriptProcessor.class.getClassLoader().getResource(defFile) == null) {
    throw new IllegalStateException("Define file not on classpath: " + defFile);
}

try {
    processor = new ScriptProcessor(language, script, threadNum);
} catch (IllegalArgumentException e) {
    if (e.getCause() instanceof java.io.IOException) {
        throw new IllegalStateException("Failed to read define file: " + language.getDefineFile(), e);
    }
    throw e;
}

When it happens

Trigger: new ScriptProcessor(language, script, threadNum) where the language's define file resource is missing from the classpath, unreadable, or the stream fails mid-read — e.g. a custom Language whose getDefineFile() points to a resource not on the classpath.

Common situations: Using a custom Language implementation without packaging its define file into the jar; fat-jar shading dropping resource files; classloader isolation (app-server/OSGi) hiding the resource; truncated/corrupt jar.

Understand the failure class

Background: 'Could not be found', 'does not exist', 'not found in database': the resource-not-found family when an ID, slug, key, or URI lookup comes back empty — this error's family across 20 libraries.

Related errors

• init cache scheduler error
• java.io.IOException
• wrong proto type map
• XPath error!
• language and script must not be null!

AI-assisted analysis of code4craft/webmagic@67816a19d6 (2026-09-08). Data as JSON: /api/errors/562182d6288dcb05. Report an issue: GitHub.

Appendix: source

Thrown at webmagic-scripts/src/main/java/us/codecraft/webmagic/scripts/ScriptProcessor.java:42

    private String defines;

    private String script;

    private final Language language;

    private Site site = Site.me();

    public ScriptProcessor(Language language, String script, int threadNum) {
        if (language == null || script == null) {
            throw new IllegalArgumentException("language and script must not be null!");
        }
        this.language = language;
        enginePool = new ScriptEnginePool(language, threadNum);
        InputStream resourceAsStream = this.getClass().getClassLoader().getResourceAsStream(language.getDefineFile());
        try {
            defines = IOUtils.toString(resourceAsStream, Charset.defaultCharset());
        } catch (IOException e) {
            throw new IllegalArgumentException(e);
        }
        this.script = script;
    }

    @Override
    public void process(Page page) {
        ScriptEngine engine = enginePool.getEngine();
        try {
            ScriptContext context = engine.getContext();
            context.setAttribute("page", page, ScriptContext.ENGINE_SCOPE);
            context.setAttribute("config", site, ScriptContext.ENGINE_SCOPE);
            try {
                this.language.process(engine, defines, script, page);
            } catch (ScriptException e) {
                e.printStackTrace();
            }
        } finally {
            enginePool.release(engine);

View on GitHub (pinned to 67816a19d6)