JuliusBrussee/caveman · error

generic decode: expected a JSON array or {"data":[...]} of o

Error message

generic decode: expected a JSON array or {"data":[...]} of objects

What it means

decodeGenericRecords accepts exactly two payload shapes: a top-level JSON array of objects, or an object with a "data" key holding an array. If json.Unmarshal fails for both shapes (or the wrapper parsed but env.Data is nil), this error is returned. It means the input bytes are not valid JSON for either accepted envelope, so mapping is never attempted.

Source

Thrown at shared/platform/importers/generic.go:123

		}
	}
	return nil
}

func decodeGenericRecords(data []byte) ([]map[string]any, error) {
	// Try a bare array first.
	var arr []map[string]any
	if err := json.Unmarshal(data, &arr); err == nil {
		return arr, nil
	}
	// Then a {"data":[...]} wrapper.
	var env struct {
		Data []map[string]any `json:"data"`
	}
	if err := json.Unmarshal(data, &env); err == nil && env.Data != nil {
		return env.Data, nil
	}
	return nil, fmt.Errorf("generic decode: expected a JSON array or {\"data\":[...]} of objects")
}

func mapGeneric(rec map[string]any, opts Options) (Span, map[string]bool, error) {
	var sp Span
	startNs, endNs := int64(0), int64(0)
	resolved := make(map[string]bool, len(opts.FieldMap))
	for target, path := range opts.FieldMap {
		kind := genericTargets[target]
		val, found := lookupPath(rec, path)
		if !found {
			if target == "trace_id" || target == "span_id" || target == "timestamp" {
				return Span{}, nil, fmt.Errorf("mapped source path %q for required target %q was not found", path, target)
			}
			continue
		}
		resolved[target] = true
		switch kind {
		case "string":

View on GitHub (pinned to 27d5a3981a)

Solutions

1. Re-shape the payload to a bare array [...] or {"data":[...]} and re-import
2. If the wrapper key is different (e.g. "rows"), unwrap it in your own code before passing bytes to the generic importer
3. For newline-delimited JSON, use the caveman-jsonl format instead of generic
4. Validate the file parses as JSON and starts with '[' or '{' before importing

Example fix

// before
raw := []byte(`{"rows":[{"traceId":"t1"}]}`)
rows, sum, err := importers.Import(importers.FormatGeneric, raw, opts)

// after
raw := []byte(`{"data":[{"traceId":"t1"}]}`)
rows, sum, err := importers.Import(importers.FormatGeneric, raw, opts)

func looksLikeGenericPayload(data []byte) bool {
	d := bytes.TrimSpace(data)
	return len(d) > 0 && (d[0] == '[' || (d[0] == '{' && bytes.Contains(bytes.ToLower(d), []byte("\"data\""))))
}

if _, _, err := importers.Import(importers.FormatGeneric, raw, opts); err != nil { if strings.HasPrefix(err.Error(), "generic decode") { return fmt.Errorf("input file is not a generic JSON export: %w", err) } }

When it happens

Trigger: Feeding a JSON object like {"rows":[...]} or {"observations":[...]} (wrong wrapper key); truncated/malformed JSON; a JSONL file (newline-delimited objects) passed to the generic importer, which expects one document; a top-level JSON scalar, string, or an array of non-object elements.

Common situations: Exporting from a vendor dashboard that wraps results in a non-standard key; piping curl output that got cut off; assuming JSONL is supported because the caveman-jsonl format exists; a UTF-8 BOM prepended by Windows tooling breaking the first Unmarshal.

Related errors

• helicone decode: %w
• langfuse decode: %w
• otlp decode: %w
• generic import requires a %q mapping
• mapped source path %q for required target %q was not found

AI-assisted analysis of JuliusBrussee/caveman@27d5a3981a (2026-08-15). Data as JSON: /api/errors/b564505c7555bd65. Report an issue: GitHub.