gastownhall/beads · error

failed to scan JSONL: %w

Error message

failed to scan JSONL: %w

What it means

The bufio.Scanner reading the JSONL stream hit an I/O or buffer error while iterating lines (not a JSON syntax problem — those fail earlier). scanner.Err() is non-nil, so the underlying read itself failed.

Source

Thrown at cmd/bd/import.go:296

					memories = append(memories, mem)
				}
				continue
			}
		}

		var issue types.Issue
		if err := json.Unmarshal([]byte(line), &issue); err != nil {
			return nil, nil, fmt.Errorf("failed to parse issue from JSONL: %w", err)
		}
		if issue.Status == "tombstone" {
			continue
		}
		applyImportWispPlane(peek, &issue)
		issue.SetDefaults()
		issues = append(issues, &issue)
	}
	if err := scanner.Err(); err != nil {
		return nil, nil, fmt.Errorf("failed to scan JSONL: %w", err)
	}
	return issues, memories, nil
}

// runImportRecordsClassic is the classic (embedded/direct store) import
// pipeline over the parsed records: dedup, dry-run classification, memory
// writes, the batch issue import, the final commit and the issue_prefix
// reconciliation.
func runImportRecordsClassic(ctx context.Context, issues []*types.Issue, memories []memoryRecord, source string) error {
	// Dedup: skip issues whose title matches an existing open issue
	dedupHits := 0
	if importDedup && len(issues) > 0 {
		issues, dedupHits = filterDuplicatesByTitle(ctx, store, issues)
	}

	result := importResultJSON{
		Source:    source,
		DedupHits: dedupHits,

View on GitHub (pinned to 71377f2769)

Solutions

1. Check that the input source (file/stdin pipe) is readable and stays available during import
2. If lines exceed ~64KB, split records or re-export with stable formatting instead of one giant line
3. Retry the pipe transfer if the producer crashed mid-stream
4. Import from a local file rather than a flaky pipe: `bd import file.jsonl`

Example fix

// before
cat huge.jsonl | bd import -   # bufio: token too long
// after
bd export -o split.jsonl   # canonical multi-line export
bd import split.jsonl

test -r data.jsonl && [ $(awk '{print length}' data.jsonl | sort -n | tail -1) -lt 60000 ]

if ! bd import data.jsonl 2>err.log; then
  grep 'failed to scan JSONL' err.log && bd import data.jsonl   # retry after source check
fi

When it happens

Trigger: `bd import -` reading from a pipe/stdin that errors mid-stream (broken pipe, closed fd), a file that disappears or has permission issues mid-read, or a single line exceeding bufio.Scanner's default 64KB max token size.

Common situations: Piping huge exports through stdin; network filesystem dropouts; extremely long single-line records produced by minified exports.

Related errors

• failed to read JSONL: %w
• reading from stdin: %w
• ErrAmbiguousID
• 'bd admin %s' is not yet supported in embedded mode
• cannot use multiple conflict resolution flags

AI-assisted analysis of gastownhall/beads@71377f2769 (2026-08-30). Data as JSON: /api/errors/39430a26a3281a6c. Report an issue: GitHub.