What causes "missing mandatory 'pattern' field in %s: %s" in crowdsecurity/crowdsec?

Open the data file named in the error, find the offending line, and add a non-empty "pattern" value.

missing mandatory 'pattern' field in

Error message

missing mandatory 'pattern' field in %s: %s

What it means

fileMapInit parses each line of a filemap data file as a JSON object and requires a 'pattern' key. This error is thrown when a JSON line has an empty or missing 'pattern' field, since the pattern is what the map is keyed on for expression lookups. The filemap cannot be initialized without it.

Solutions

Open the data file named in the error, find the offending line, and add a non-empty "pattern" value.
Validate every JSON line in the file with a script (jq -c each line, check .pattern != "") before loading it.
Re-download or regenerate the data file from the upstream hub list if it was corrupted by editing or truncation.
Remove blank or malformed lines that only carry whitespace/placeholder content.

Example fix

// before (data file line)
{"tag":"bad-bot"}
// after
{"pattern":"8.8.8.8","tag":"bad-bot","type":"ip"}

Defensive patterns

Strategy: validation

Validate before calling

for i, line := range lines {
	var rec map[string]string
	if err := json.Unmarshal([]byte(line), &rec); err != nil { /* skip */ }
	if rec["pattern"] == "" {
		return fmt.Errorf("line %d: missing 'pattern'", i+1)
	}
}

Prevention

Validate every data-file line with jq before loading (jq -c 'select(.pattern == "" or .pattern == null)')
Keep data files machine-generated from a single source of truth rather than hand-editing
Add filemap schema checks to CI for custom data files

When it happens

Trigger: Calling FileInit (via fileMapInit) on a filemap where a line decodes to valid JSON but lacks the 'pattern' key or has it set to an empty string.

Common situations: Hand-edited .txt data files for MapType (e.g. from a hublist or lookup table) where a row was added with only a tag, or where the pattern column was left blank; files generated by scripts that emit partial records.

Understand the failure class

Background: "is required", "must be set", "missing required field": configuration validation errors across open-source libraries — this error's family across 36 libraries.

Related errors

AI-assisted analysis of crowdsecurity/crowdsec@909b515798 (2026-09-06). Data as JSON: /api/errors/47df83bb901d1da4. Report an issue: GitHub.

Appendix: source

Thrown at pkg/exprhelpers/filemap.go:51

	// Aho-Corasick automaton for "contains" entries (O(|haystack|) matching, checked second).
	acAutomaton    *aho_corasick.AhoCorasick
	acPatternToRow []int // AC pattern index → row index in fileMapEntry.rows

	// Pre-compiled regexps for "regex" entries (checked last).
	regexPatterns []*regexp.Regexp
	regexToRow    []int // regex slice index → row index in fileMapEntry.rows
}

// fileMapInit parses a single JSON line and appends it to the fileMapEntry for the given filename.
// Three fields are mandatory: "pattern", "tag", and "type" (one of: "equals", "contains", "regex").
func fileMapInit(filename string, line string) error {
	var record map[string]string
	if err := json.Unmarshal([]byte(line), &record); err != nil {
		return fmt.Errorf("failed to parse JSON line in %s: %w", filename, err)
	}

	if record["pattern"] == "" {
		return fmt.Errorf("missing mandatory 'pattern' field in %s: %s", filename, line)
	}

	if record["tag"] == "" {
		return fmt.Errorf("missing mandatory 'tag' field in %s: %s", filename, line)
	}

	entryType := record["type"]
	if entryType == "" {
		return fmt.Errorf("missing mandatory 'type' field in %s: %s", filename, line)
	}

	if !slices.Contains(validMapEntryTypes, entryType) {
		return fmt.Errorf("unknown entry type '%s' in %s (supported: %s): %s",
			entryType, filename, strings.Join(validMapEntryTypes, ", "), line)
	}

	if entryType == "regex" {
		if _, err := regexp.Compile(record["pattern"]); err != nil {

View on GitHub (pinned to 909b515798)