projectdiscovery/katana · error

error compiling pattern %s: %v

Error message

error compiling pattern %s: %v

What it means

NewTextNormalizer compiles each caller-supplied regex pattern with regexp.Compile and fails fast, wrapping the compile error with the offending pattern. It prevents constructing a TextNormalizer with invalid patterns, since compilation errors would otherwise surface per-use at runtime.

Source

Thrown at pkg/engine/headless/crawler/normalizer/text_utils.go:48

type TextNormalizer struct {
	// patterns is a list of regex patterns for the text normalizer
	patterns []*regexp.Regexp
}

// NewTextNormalizer returns a new TextNormalizer
//
// patterns is a list of regex patterns for the text normalizer
// DefaultTextPatterns is used if patterns is nil. See DefaultTextPatterns for more info.
func NewTextNormalizer() (*TextNormalizer, error) {
	patterns := slices.Clone(DefaultTextPatterns)
	patterns = append(patterns, dateTimePatterns...)

	var compiledPatterns []*regexp.Regexp
	for _, pattern := range patterns {
		pattern := pattern
		compiledPattern, err := regexp.Compile(pattern)
		if err != nil {
			return nil, fmt.Errorf("error compiling pattern %s: %v", pattern, err)
		}
		compiledPatterns = append(compiledPatterns, compiledPattern)
	}
	return &TextNormalizer{patterns: compiledPatterns}, nil
}

// Apply applies the patterns to the text and returns the normalized text
func (n *TextNormalizer) Apply(text string) string {
	for _, pattern := range n.patterns {
		pattern := pattern
		text = pattern.ReplaceAllString(text, "")
	}
	return text
}

View on GitHub (pinned to e3e742739c)

Solutions

1. Read the wrapped error and offending pattern from the message to locate the syntax problem
2. Validate patterns with regexp.Compile in a preflight/config-load step and report which entry failed
3. Rewrite PCRE-only constructs in RE2-compatible form (e.g. replace lookaheads with explicit matching)
4. Test custom patterns against regexp.Compile or an RE2 linter before deployment
5. Keep patterns simple; prefer literal string handling for non-regex normalization needs

Example fix

// before
patterns := []string{"(?<=user_id=)\d+"} // lookbehind unsupported in RE2

// after
patterns := []string{"user_id=\d+"} // RE2-compatible

for _, p := range patterns {
	if _, err := regexp.Compile(p); err != nil {
		return fmt.Errorf("invalid normalizer pattern %q: %w", p, err)
	}
}

norm, err := normalizer.NewTextNormalizer(patterns)
if err != nil {
	return nil, fmt.Errorf("normalizer init failed: %w", err)
}

When it happens

Trigger: Called by New (and tests) when a pattern in the patterns slice is not valid RE2 syntax — e.g. unbalanced parentheses, invalid escape sequences, or Perl-only constructs that Go's regexp rejects.

Common situations: User-supplied normalization patterns from config files that contain PCRE-specific syntax like lookaheads ((?=...)), backreferences (\1), or possessive quantifiers, none of which Go's RE2 supports; truncated patterns; escaped-character mistakes.

Related errors

• response filtered by similarity detection
• result does not match extension filter
• result does not match output
• result is filtered out
• result filtered by page type

AI-assisted analysis of projectdiscovery/katana@e3e742739c (2026-09-03). Data as JSON: /api/errors/28e6eda504c92e21. Report an issue: GitHub.