{"record":{"id":"e709c16c574fcf08","repo":"JuliusBrussee/caveman","slug":"cachebench-observation-line-d-duplicate-or-inva","errorCode":null,"errorMessage":"cachebench: observation line %d: duplicate or invalid JSON","messagePattern":"cachebench: observation line (.+?): duplicate or invalid JSON","errorType":"validation","errorClass":null,"httpStatus":null,"severity":"error","filePath":"cacheengine/cachebench/observed.go","lineNumber":116,"sourceCode":"\t}\n\tscanner := bufio.NewScanner(reader)\n\tinitial := 64 * 1024\n\tif limits.MaxLineBytes < initial {\n\t\tinitial = limits.MaxLineBytes\n\t}\n\tscanner.Buffer(make([]byte, initial), limits.MaxLineBytes)\n\tseen := map[string]bool{}\n\tvar records []ObservationRecord\n\tfor line := 1; scanner.Scan(); line++ {\n\t\traw := bytes.TrimSpace(scanner.Bytes())\n\t\tif len(raw) == 0 {\n\t\t\tcontinue\n\t\t}\n\t\tif len(records) >= limits.MaxRecords {\n\t\t\treturn nil, fmt.Errorf(\"cachebench: observations exceed record limit %d\", limits.MaxRecords)\n\t\t}\n\t\tif !validUniqueJSONObject(raw) {\n\t\t\treturn nil, fmt.Errorf(\"cachebench: observation line %d: duplicate or invalid JSON\", line)\n\t\t}\n\t\tdecoder := json.NewDecoder(bytes.NewReader(raw))\n\t\tdecoder.DisallowUnknownFields()\n\t\tvar record ObservationRecord\n\t\tif err := decoder.Decode(&record); err != nil {\n\t\t\treturn nil, fmt.Errorf(\"cachebench: observation line %d: %w\", line, err)\n\t\t}\n\t\tvar trailing any\n\t\tif err := decoder.Decode(&trailing); err != io.EOF {\n\t\t\treturn nil, fmt.Errorf(\"cachebench: observation line %d: trailing JSON\", line)\n\t\t}\n\t\tif record.Schema != ObservationSchema {\n\t\t\treturn nil, fmt.Errorf(\"cachebench: observation line %d: schema %q\", line, record.Schema)\n\t\t}\n\t\tif strings.TrimSpace(record.RequestID) == \"\" || seen[record.RequestID] {\n\t\t\treturn nil, fmt.Errorf(\"cachebench: observation line %d: empty or duplicate request_id\", line)\n\t\t}\n\t\tseen[record.RequestID] = true","sourceCodeStart":98,"sourceCodeEnd":134,"githubUrl":"https://github.com/JuliusBrussee/caveman/blob/27d5a3981a347890211bb1bf2439e5c821a63bc9/cacheengine/cachebench/observed.go#L98-L134","documentation":"An observation line failed validUniqueJSONObject — it must be a single valid JSON object with no duplicate keys. The check runs before decoding so malformed or ambiguous lines (duplicate keys would make decoding order-dependent) are rejected deterministically with the 1-based line number.","triggerScenarios":"Reading observations where line N is not a JSON object (array, bare string, truncated JSON) or contains repeated keys like two 'request_id' members.","commonSituations":"Log shipping that truncates long lines; observation emitters that merge maps and duplicate keys; files with BOM or garbage interleaved into lines.","solutions":["Fix or remove the malformed line identified by the line number.","Re-emit observations from the source tool rather than post-processing with scripts that can duplicate keys.","Validate the file with a strict JSON linter (duplicate-key detection) before loading."],"exampleFix":null,"handlingStrategy":"validation","validationCode":"func validateObservationFile(path string) error {\n\tdata, err := os.ReadFile(path)\n\tif err != nil {\n\t\treturn err\n\t}\n\tfor i, line := range strings.Split(string(data), \"\\n\") {\n\t\tline = strings.TrimSpace(line)\n\t\tif line == \"\" {\n\t\t\tcontinue\n\t\t}\n\t\tif !validUniqueJSONObject([]byte(line)) {\n\t\t\treturn fmt.Errorf(\"observation line %d: duplicate or invalid JSON\", i+1)\n\t\t}\n\t}\n\treturn nil\n}","typeGuard":"func isJSONObject(b []byte) bool {\n\tt := bytes.TrimSpace(b)\n\treturn len(t) > 0 && t[0] == '{' && json.Valid(t)\n}","tryCatchPattern":null,"preventionTips":["Write observations with encoding/json Marshal (one object per line).","Avoid post-processing observation files with tools that merge keys.","Check for line truncation if logs pass through size-limited shippers."],"tags":["go","json","jsonl","validation"],"backgroundTag":null,"analyzedSha":"27d5a3981a347890211bb1bf2439e5c821a63bc9","analyzedAt":"2026-08-15T09:26:11.751Z","schemaVersion":2},"datasetVersion":"2026-08-15T22:17:37.221Z"}