What causes "failed to read JSONL: %w" in gastownhall/beads?

Re-run the check when no other bd process is writing (stop concurrent sync/export first).

failed to read JSONL: %w

Error message

failed to read JSONL: %w

What it means

While streaming the JSONL file line-by-line with a bufio.Scanner, validateJSONLForMigration checks scanner.Err() after the loop. A non-nil error (I/O failure mid-read, not a parse problem) is wrapped as 'failed to read JSONL'. Unlike malformed lines, which are counted, this is a hard read failure of an already-opened file.

Source

Thrown at cmd/bd/doctor/migration_validation.go:429

			if len(parseErrors) < 5 {
				parseErrors = append(parseErrors, fmt.Sprintf("line %d: %v", lineNo, err))
			}
			continue
		}

		if issue.ID == "" {
			malformed++
			if len(parseErrors) < 5 {
				parseErrors = append(parseErrors, fmt.Sprintf("line %d: missing id field", lineNo))
			}
			continue
		}

		ids[issue.ID] = true
	}

	if err := scanner.Err(); err != nil {
		return len(ids), malformed, ids, fmt.Errorf("failed to read JSONL: %w", err)
	}

	// Return error only if ALL lines are malformed (blocking)
	if len(ids) == 0 && malformed > 0 {
		return 0, malformed, ids, fmt.Errorf("JSONL file is completely corrupt: %d malformed lines", malformed)
	}

	return len(ids), malformed, ids, nil
}

// compareDoltWithJSONL compares Dolt database with JSONL IDs.
// Returns IDs in JSONL but not in Dolt (sample first 100).
func compareDoltWithJSONL(ctx context.Context, store storage.DoltStorage, jsonlIDs map[string]bool) []string {
	ids := make([]string, 0, len(jsonlIDs))
	for id := range jsonlIDs {
		ids = append(ids, id)
	}
	if len(ids) == 0 {

View on GitHub (pinned to 71377f2769)

Solutions

Re-run the check when no other bd process is writing (stop concurrent sync/export first).
If the error is 'token too long', the JSONL has an oversized line — re-export with bd sync and inspect the offending line.
Check disk/filesystem health (dmesg, df) if I/O errors repeat.
Restore a good JSONL from the git remote: bd dolt pull && bd sync, then retry.

Example fix

// before
Error: failed to read JSONL: bufio.Scanner: token too long
// after
$ bd sync   # regenerate a clean issues.jsonl
$ bd doctor migrate-check

Defensive patterns

Strategy: retry

Validate before calling

f, err := os.Open(jsonlPath)
if err != nil { return err }
sc := bufio.NewScanner(f)
sc.Buffer(make([]byte, 0, 1024*1024), 10*1024*1024) // allow very long lines
_ = sc; f.Close()

Try / catch

valid, malformed, ids, err := validateJSONLForMigration(jsonlPath)
if err != nil && strings.Contains(err.Error(), "failed to read JSONL") {
    time.Sleep(time.Second)          // let a concurrent writer finish
    valid, malformed, ids, err = validateJSONLForMigration(jsonlPath)
}

Prevention

Do not run doctor migration checks while bd sync/export is writing the JSONL.
Re-export after interrupted syncs to avoid truncated files.
Use a larger scanner buffer when lines may exceed 64KB.
Check disk/network filesystem health if read errors recur.

When it happens

Trigger: scanner.Err() is non-nil after scanning — e.g. the file was truncated/modified while being read, a disk I/O error occurred, or (rarely) scanner buffer limits on an extremely long single line.

Common situations: A concurrent bd process rewriting issues.jsonl during doctor's migration check; failing disk or network filesystem; a single JSONL line exceeding the default 64KB scanner token limit (bufio.Scanner: token too long).

Related errors

AI-assisted analysis of gastownhall/beads@71377f2769 (2026-08-30). Data as JSON: /api/errors/cf8d765506db8003. Report an issue: GitHub.