What causes "caveman-jsonl line %d: %w" in JuliusBrussee/caveman?

Open the file at the reported line number and inspect it: head -n N+1 file | tail -1.

caveman-jsonl line %d: %w

Error message

caveman-jsonl line %d: %w

What it means

Thrown by parseCavemanJSONL (shared/platform/importers/caveman_jsonl.go:22) when a non-blank line fails json.Unmarshal into the Span shape. The importer deliberately aborts the whole file on the first malformed line - the doc comment states 'no partial silent ingest' - so a single bad line invalidates the batch, and the 1-based line number identifies it.

Source

Thrown at shared/platform/importers/caveman_jsonl.go:22

	"bytes"
	"encoding/json"
	"fmt"
)

// parseCavemanJSONL parses one JSON span per line. Each line is already close
// to the Span shape (the same column names), so it unmarshals directly into a
// Span and then has the authenticated scope stamped over it. Blank lines are
// skipped. A malformed line aborts the whole import (no partial silent ingest).
func parseCavemanJSONL(data []byte, opts Options) ([]Span, error) {
	var rows []Span
	for i, line := range bytes.Split(data, []byte("\n")) {
		line = bytes.TrimSpace(line)
		if len(line) == 0 {
			continue
		}
		var sp Span
		if err := json.Unmarshal(line, &sp); err != nil {
			return nil, fmt.Errorf("caveman-jsonl line %d: %w", i+1, err)
		}
		applyScope(&sp, opts)
		rows = append(rows, sp)
	}
	return rows, nil
}

View on GitHub (pinned to 27d5a3981a)

Solutions

Open the file at the reported line number and inspect it: head -n N+1 file | tail -1.
Validate every line quickly: awk 'NF{n++}END{print n}' plus jq -c . file > /dev/null to locate all offenders, not just the first.
Remove or fix the malformed lines, then re-run the import (previous attempt ingested nothing).
If a writer is truncating lines, fix its flush/atomic-write behavior (write temp + rename) so imports get complete files.

Example fix

# before (line 42 broken)
{"span_id":"a1"..."trace_id":"t"

# after
{"span_id":"a1","trace_id":"t"}

Defensive patterns

Strategy: validation

Validate before calling

// pre-flight the whole file line by line
for i, line := range bytes.Split(data, []byte("\n")) {
    line = bytes.TrimSpace(line)
    if len(line) == 0 { continue }
    if !json.Valid(line) {
        return fmt.Errorf("line %d is not valid JSON; import aborted by design", i+1)
    }
}

Type guard

func validJSONL(data []byte) bool {
    for _, line := range bytes.Split(data, []byte("\n")) {
        line = bytes.TrimSpace(line)
        if len(line) > 0 && !json.Valid(line) { return false }
    }
    return true
}

Try / catch

spans, err := importers.ImportCavemanJSONL(data, opts)
if err != nil {
    return fmt.Errorf("import rejected (nothing ingested): %w", err) // do not retry the same bytes
}

Prevention

Validate JSONL with a jq pass before importing.
Have producers write files atomically so importers never read partial lines.
Keep exporter output homogeneous - one record shape per file.

When it happens

Trigger: Importing a .jsonl file where some line is not valid JSON, or is valid JSON whose shape conflicts with the Span struct (wrong types, e.g. a string where a number is expected). Blank/whitespace-only lines are skipped and never trigger this; only lines that attempt unmarshal can fail.

Common situations: Log rotation interleaving partial writes into the file; a producer crashing mid-line leaving one truncated JSON line; jq/awk post-processing that dropped a closing brace; mixed formats in one file (CSV header row, exporter banner); NaN/Infinity values that JSON forbids.

Related errors

AI-assisted analysis of JuliusBrussee/caveman@27d5a3981a (2026-08-15). Data as JSON: /api/errors/a0d6b3cc72ff1c93. Report an issue: GitHub.