projectdiscovery/katana · error

could not compile regex %s: %s

Error message

could not compile regex %s: %s

What it means

NewManager validates the field-scope value against known DNS scope fields; if it isn't a predefined field, it is treated as a custom regex and compiled. regexp.Compile failing on a malformed pattern produces this error and aborts manager creation. The pattern text and the regexp error are both embedded in the message.

Source

Thrown at pkg/utils/scope/scope.go:46

	customDNSScopeField
)

var stringToDNSScopeField = map[string]dnsScopeField{
	"dn":   dnDnsScopeField,
	"rdn":  rdnDnsScopeField,
	"fqdn": fqdnDNSScopeField,
}

// NewManager returns a new scope manager for crawling
func NewManager(inScope, outOfScope []string, fieldScope string, noScope bool) (*Manager, error) {
	manager := &Manager{
		noScope: noScope,
	}

	if scopeValue, ok := stringToDNSScopeField[fieldScope]; !ok {
		manager.fieldScope = customDNSScopeField
		if compiled, err := regexp.Compile(fieldScope); err != nil {
			return nil, fmt.Errorf("could not compile regex %s: %s", fieldScope, err)
		} else {
			manager.fieldScopePattern = compiled
		}
	} else {
		manager.fieldScope = scopeValue
	}
	for _, regex := range inScope {
		if compiled, err := regexp.Compile(regex); err != nil {
			return nil, fmt.Errorf("could not compile regex %s: %s", regex, err)
		} else {
			manager.inScope = append(manager.inScope, compiled)
		}
	}
	for _, regex := range outOfScope {
		if compiled, err := regexp.Compile(regex); err != nil {
			return nil, fmt.Errorf("could not compile regex %s: %s", regex, err)
		} else {
			manager.outOfScope = append(manager.outOfScope, compiled)

View on GitHub (pinned to e3e742739c)

Solutions

1. Fix the fieldScope regex syntax — test it with https://regex101.com (Go flavor) or regexp.Compile in a scratch program.
2. Replace glob-style patterns (*.) with regex equivalents (e.g., "^([a-z0-9-]+\\.)*corp\\.local$").
3. Remove unsupported RE2 constructs such as lookaheads/lookbehinds and backreferences.
4. Use a predefined fieldScope value (a known DNS scope field) instead of a custom regex if one matches your intent.

Example fix

// before
mgr, err := scope.NewManager("*.corp.local", nil, nil) // glob, not regex
// after
mgr, err := scope.NewManager("^([a-z0-9-]+\\.)*corp\\.local$", nil, nil)

if _, err := regexp.Compile(fieldScope); err != nil {
    return fmt.Errorf("invalid field scope pattern %q: %w", fieldScope, err)
}

func isValidRegex(s string) bool {
    _, err := regexp.Compile(s)
    return err == nil
}

mgr, err := scope.NewManager(fieldScope, inScope, outOfScope)
if err != nil {
    if strings.Contains(err.Error(), "could not compile regex") {
        // surface the pattern to the user with config file/line context before exiting
    }
}

When it happens

Trigger: Calling scope.NewManager(fieldScope, inScope, outOfScope) with a fieldScope string that is neither a known DNS scope field (e.g., "domain", "full") nor a valid Go regexp — e.g., "[a-z" or "*.example.com".

Common situations: Reading scope config from YAML/flags where the user confused glob syntax with regex ("*.corp.local"); forgetting Go regexp is RE2 (no lookaheads like (?!...)); accidental shell or JSON escaping mangling backslashes.

Related errors

• error compiling pattern %s: %v
• unknown similarity mode %q (want simhash, tfidf, or bm25)
• publicsuffix: empty label in domain %q

AI-assisted analysis of projectdiscovery/katana@e3e742739c (2026-09-03). Data as JSON: /api/errors/75345af429604178. Report an issue: GitHub.