What causes "invalid regex " in code4craft/webmagic?

Fix the regex syntax — check the cause PatternSyntaxException message for the position of the error

code4craft/webmagic · error · IllegalArgumentException

invalid regex

Error message

invalid regex

What it means

compileRegex wraps Pattern.compile failures: when the supplied string is not a valid Java regular expression, PatternSyntaxException is rethrown as IllegalArgumentException("invalid regex " + regexStr, e). Note the thrown message plus the cause carries the offending pattern.

Solutions

Fix the regex syntax — check the cause PatternSyntaxException message for the position of the error
Test the pattern with Pattern.compile(regex) in isolation or a regex debugger before wiring it in
Escape metacharacters (Pattern.quote) when the pattern is built from user/config text

Example fix

// before
new RegexSelector("[(.*?)]");
// after
new RegexSelector("\\[(.*?)\\]");

Defensive patterns

Strategy: validation

Validate before calling

try { java.util.regex.Pattern.compile(regexStr); } catch (PatternSyntaxException e) { /* reject before building selector */ }

Try / catch

try { sel = new RegexSelector(userRegex); } catch (IllegalArgumentException e) { log.error("invalid regex: " + e.getCause(), e); }

Prevention

Unit-test every regex with Pattern.compile
Escape user input with Pattern.quote
Stick to Java regex syntax when porting patterns

When it happens

Trigger: new RegexSelector("[") or any syntactically invalid pattern (unclosed group, dangling quantifier, bad escape); also ReplaceSelector misuse aside, this is RegexSelector's path.

Common situations: Patterns written for another flavor (e.g. PCRE/JS) using unsupported constructs; hand-edited regex with a typo; dynamically built regex where user input breaks syntax.

Related errors

AI-assisted analysis of code4craft/webmagic@67816a19d6 (2026-09-08). Data as JSON: /api/errors/85c493c0b4d642ca. Report an issue: GitHub.

Appendix: source

Thrown at webmagic-core/src/main/java/us/codecraft/webmagic/selector/RegexSelector.java:38

    private Pattern regex;

    private int group = 1;

    public RegexSelector(String regexStr, int group) {
        this.compileRegex(regexStr);
        this.group = group;
    }

    private void compileRegex(String regexStr) {
        if (StringUtils.isBlank(regexStr)) {
            throw new IllegalArgumentException("regex must not be empty");
        }
        try {
            this.regex = Pattern.compile(regexStr, Pattern.DOTALL | Pattern.CASE_INSENSITIVE);
            this.regexStr = regexStr;
        } catch (PatternSyntaxException e) {
            throw new IllegalArgumentException("invalid regex "+regexStr, e);
        }
    }

    /**
     * Create a RegexSelector. When there is no capture group, the value is set to 0 else set to 1.
     * @param regexStr the regular expression.
     */
    public RegexSelector(String regexStr) {
        this.compileRegex(regexStr);
        if (regex.matcher("").groupCount() == 0) {
            this.group = 0;
        } else {
            this.group = 1;
        }
    }

    @Override
    public String select(String text) {

View on GitHub (pinned to 67816a19d6)