{"record":{"id":"31771648fd405c5f","repo":"crowdsecurity/crowdsec","slug":"failed-to-parse-json-line-in-s-w","errorCode":null,"errorMessage":"failed to parse JSON line in %s: %w","messagePattern":"failed to parse JSON line in (.+?): %w","errorType":"validation","errorClass":null,"httpStatus":null,"severity":"error","filePath":"pkg/exprhelpers/botfile.go","lineNumber":55,"sourceCode":"\tuaRegex     *regexp.Regexp\n\tpathRegexes []*regexp.Regexp\n\tipSet       map[netip.Addr]struct{}\n\tprefixes    []netip.Prefix\n\trdnsRegexes []*regexp.Regexp\n}\n\nfunc compileBotRegex(pattern string) (*regexp.Regexp, error) {\n\treturn regexp.Compile(\"(?i)\" + pattern) // Force case insensitive match\n}\n\nfunc botFileInit(filename string, line string) error {\n\tentry := &botEntry{}\n\n\tdec := json.NewDecoder(strings.NewReader(line))\n\tdec.DisallowUnknownFields()\n\n\tif err := dec.Decode(entry); err != nil {\n\t\treturn fmt.Errorf(\"failed to parse JSON line in %s: %w\", filename, err)\n\t}\n\n\tif entry.Name == \"\" {\n\t\treturn fmt.Errorf(\"missing mandatory 'name' field in %s: %s\", filename, line)\n\t}\n\n\tif len(entry.IPs)+len(entry.Ranges)+len(entry.RDNS) == 0 {\n\t\treturn fmt.Errorf(\"bot entry '%s' in %s has no identity verification (need at least one of ips/ranges/rdns)\", entry.Name, filename)\n\t}\n\n\tvar err error\n\n\tif entry.UserAgent != \"\" {\n\t\tif entry.uaRegex, err = compileBotRegex(entry.UserAgent); err != nil {\n\t\t\treturn fmt.Errorf(\"invalid user_agent regex for bot entry '%s' in %s: %w\", entry.Name, filename, err)\n\t\t}\n\t}\n","sourceCodeStart":37,"sourceCodeEnd":73,"githubUrl":"https://github.com/crowdsecurity/crowdsec/blob/909b5157986a2b2c2163300fdaef5ed01289f7d2/pkg/exprhelpers/botfile.go#L37-L73","documentation":"botFileInit parses each line of a 'bots' JSON-lines data file into a botEntry struct with DisallowUnknownFields enabled. This error is returned when the line is not valid JSON at all, or contains fields not defined on botEntry. It is thrown during data-file loading (FileInit) so that malformed bot data fails fast at startup instead of silently mis-matching at runtime.","triggerScenarios":"A line in the bots data file is not valid JSON (trailing comma, single quotes, truncation, BOM), or uses a field name not in the botEntry schema (name, user_agent, paths, ips, ranges, rdns) — e.g. 'ip' instead of 'ips' or 'useragent' instead of 'user_agent'.","commonSituations":"Hand-edited or copy-pasted bot entries with JSON syntax typos; converting a CSV/YAML bot list to JSONL incorrectly; upgrading CrowdSec after a schema change and keeping an old file with renamed fields; a build/template step emitting two JSON objects on one line.","solutions":["Validate the line with a JSON linter or `echo '<line>' | jq .` and fix the syntax error indicated by the wrapped %w error (it names the exact byte offset).","Check for unknown field names: only name, user_agent, paths, ips, ranges, rdns are accepted — rename any other key or remove it.","Ensure the file is strict JSON-lines: one JSON object per line, no commas between lines, no BOM, no blank malformed lines.","If the file came from an external source or older CrowdSec version, re-download the current format or migrate field names.","Run `cscli` or the FileInit path again after fixing; the error message includes the filename so fix that specific file."],"exampleFix":"// before (unknown field + trailing comma)\n{\"name\":\"mybot\",\"ip\":\"1.2.3.4\",}\n// after\n{\"name\":\"mybot\",\"ips\":[\"1.2.3.4\"]}","handlingStrategy":"validation","validationCode":"// pre-validate a bot file line before handing it to the loader\nfunc validBotLine(line string) bool {\n\tvar raw map[string]json.RawMessage\n\tif err := json.Unmarshal([]byte(line), &raw); err != nil {\n\t\treturn false\n\t}\n\tallowed := map[string]bool{\"name\": true, \"user_agent\": true, \"paths\": true, \"ips\": true, \"ranges\": true, \"rdns\": true}\n\tfor k := range raw {\n\t\tif !allowed[k] {\n\t\t\treturn false\n\t\t}\n\t}\n\treturn raw[\"name\"] != nil\n}","typeGuard":null,"tryCatchPattern":"if err := exprhelpers.FileInit(botFile, \"bots\"); err != nil {\n\tvar lineno int\n\t// the loader reports the offending line/file; log and abort startup\n\tlog.Fatalf(\"bot data file rejected: %v\", err)\n}","preventionTips":["Lint every data file with jq before deployment: `jq -c . file.jsonl` rejects invalid lines.","Validate field names against the documented schema (name, user_agent, paths, ips, ranges, rdns).","Generate entries programmatically with json.Marshal instead of string concatenation.","Strip BOM and CRLF when producing files on Windows.","Add a CI check that parses all bot data files with the same DisallowUnknownFields decoder."],"tags":["go","json","data-files","crowdsec"],"backgroundTag":"json-unmarshal-failed","analyzedSha":"909b5157986a2b2c2163300fdaef5ed01289f7d2","analyzedAt":"2026-09-06T12:27:26.012Z","contentChangedAt":"2026-09-06T12:27:26.012Z","schemaVersion":2},"datasetVersion":"2026-09-14T05:17:10.506Z"}