{"record":{"id":"cf8d765506db8003","repo":"gastownhall/beads","slug":"failed-to-read-jsonl-w","errorCode":null,"errorMessage":"failed to read JSONL: %w","messagePattern":"failed to read JSONL: %w","errorType":"console","errorClass":null,"httpStatus":null,"severity":"error","filePath":"cmd/bd/doctor/migration_validation.go","lineNumber":429,"sourceCode":"\t\t\tif len(parseErrors) < 5 {\n\t\t\t\tparseErrors = append(parseErrors, fmt.Sprintf(\"line %d: %v\", lineNo, err))\n\t\t\t}\n\t\t\tcontinue\n\t\t}\n\n\t\tif issue.ID == \"\" {\n\t\t\tmalformed++\n\t\t\tif len(parseErrors) < 5 {\n\t\t\t\tparseErrors = append(parseErrors, fmt.Sprintf(\"line %d: missing id field\", lineNo))\n\t\t\t}\n\t\t\tcontinue\n\t\t}\n\n\t\tids[issue.ID] = true\n\t}\n\n\tif err := scanner.Err(); err != nil {\n\t\treturn len(ids), malformed, ids, fmt.Errorf(\"failed to read JSONL: %w\", err)\n\t}\n\n\t// Return error only if ALL lines are malformed (blocking)\n\tif len(ids) == 0 && malformed > 0 {\n\t\treturn 0, malformed, ids, fmt.Errorf(\"JSONL file is completely corrupt: %d malformed lines\", malformed)\n\t}\n\n\treturn len(ids), malformed, ids, nil\n}\n\n// compareDoltWithJSONL compares Dolt database with JSONL IDs.\n// Returns IDs in JSONL but not in Dolt (sample first 100).\nfunc compareDoltWithJSONL(ctx context.Context, store storage.DoltStorage, jsonlIDs map[string]bool) []string {\n\tids := make([]string, 0, len(jsonlIDs))\n\tfor id := range jsonlIDs {\n\t\tids = append(ids, id)\n\t}\n\tif len(ids) == 0 {","sourceCodeStart":411,"sourceCodeEnd":447,"githubUrl":"https://github.com/gastownhall/beads/blob/71377f276968b452ee607177637970a4ff888584/cmd/bd/doctor/migration_validation.go#L411-L447","documentation":"While streaming the JSONL file line-by-line with a bufio.Scanner, validateJSONLForMigration checks scanner.Err() after the loop. A non-nil error (I/O failure mid-read, not a parse problem) is wrapped as 'failed to read JSONL'. Unlike malformed lines, which are counted, this is a hard read failure of an already-opened file.","triggerScenarios":"scanner.Err() is non-nil after scanning — e.g. the file was truncated/modified while being read, a disk I/O error occurred, or (rarely) scanner buffer limits on an extremely long single line.","commonSituations":"A concurrent bd process rewriting issues.jsonl during doctor's migration check; failing disk or network filesystem; a single JSONL line exceeding the default 64KB scanner token limit (bufio.Scanner: token too long).","solutions":["Re-run the check when no other bd process is writing (stop concurrent sync/export first).","If the error is 'token too long', the JSONL has an oversized line — re-export with bd sync and inspect the offending line.","Check disk/filesystem health (dmesg, df) if I/O errors repeat.","Restore a good JSONL from the git remote: bd dolt pull && bd sync, then retry."],"exampleFix":"// before\nError: failed to read JSONL: bufio.Scanner: token too long\n// after\n$ bd sync   # regenerate a clean issues.jsonl\n$ bd doctor migrate-check","handlingStrategy":"retry","validationCode":"f, err := os.Open(jsonlPath)\nif err != nil { return err }\nsc := bufio.NewScanner(f)\nsc.Buffer(make([]byte, 0, 1024*1024), 10*1024*1024) // allow very long lines\n_ = sc; f.Close()","typeGuard":null,"tryCatchPattern":"valid, malformed, ids, err := validateJSONLForMigration(jsonlPath)\nif err != nil && strings.Contains(err.Error(), \"failed to read JSONL\") {\n    time.Sleep(time.Second)          // let a concurrent writer finish\n    valid, malformed, ids, err = validateJSONLForMigration(jsonlPath)\n}","preventionTips":["Do not run doctor migration checks while bd sync/export is writing the JSONL.","Re-export after interrupted syncs to avoid truncated files.","Use a larger scanner buffer when lines may exceed 64KB.","Check disk/network filesystem health if read errors recur."],"tags":["jsonl","io","scanner","migration"],"backgroundTag":"jsonl-read-failed","analyzedSha":"71377f276968b452ee607177637970a4ff888584","analyzedAt":"2026-08-30T18:55:39.744Z","schemaVersion":2},"datasetVersion":"2026-08-30T23:17:21.991Z"}