JuliusBrussee/caveman · error

mapped source path %q for required target %q was not found

Error message

mapped source path %q for required target %q was not found

What it means

During mapGeneric, the source path registered in FieldMap for a required target (trace_id, span_id, or timestamp) was resolved against the record with lookupPath and came back not-found. Required targets must resolve on every record, unlike optional targets which are silently skipped. The error names both the missing path and the target so you can see which mapping and which record field disagreed.

Source

Thrown at shared/platform/importers/generic.go:135

	var env struct {
		Data []map[string]any `json:"data"`
	}
	if err := json.Unmarshal(data, &env); err == nil && env.Data != nil {
		return env.Data, nil
	}
	return nil, fmt.Errorf("generic decode: expected a JSON array or {\"data\":[...]} of objects")
}

func mapGeneric(rec map[string]any, opts Options) (Span, map[string]bool, error) {
	var sp Span
	startNs, endNs := int64(0), int64(0)
	resolved := make(map[string]bool, len(opts.FieldMap))
	for target, path := range opts.FieldMap {
		kind := genericTargets[target]
		val, found := lookupPath(rec, path)
		if !found {
			if target == "trace_id" || target == "span_id" || target == "timestamp" {
				return Span{}, nil, fmt.Errorf("mapped source path %q for required target %q was not found", path, target)
			}
			continue
		}
		resolved[target] = true
		switch kind {
		case "string":
			assignString(&sp, target, asString(val))
		case "int":
			assignInt(&sp, target, parseFlexInt(val))
		case "float":
			sp.TotalCostUSD = roundUSD(parseFlexFloat(val))
		case "time":
			s, ns := parseTimeFlexible(val)
			if target == "timestamp" {
				sp.Timestamp = s
				startNs = ns
			} else {
				sp.EndTimestamp = s

View on GitHub (pinned to 27d5a3981a)

Solutions

1. Inspect the actual record JSON and correct the source path in FieldMap to the real key (watch case and nesting)
2. Verify lookupPath semantics: dotted paths traverse nested objects, so ensure each intermediate segment exists on every record
3. Pre-scan the data for records missing the required paths and fix or drop them before import
4. If the field is genuinely absent per-record, that record cannot be imported by this format — enrich it upstream

Example fix

// before
opts.FieldMap = map[string]string{
    "trace_id":  "traceId",
    "span_id":   "spanId",
    "timestamp": "startTime", // records actually use "started_at"
}

// after
opts.FieldMap = map[string]string{
    "trace_id":  "traceId",
    "span_id":   "spanId",
    "timestamp": "started_at",
}

func verifyPaths(records []map[string]any, fm map[string]string) error {
	for _, req := range []string{"trace_id", "span_id", "timestamp"} {
		ok := false
		for _, rec := range records {
			if _, found := lookupPath(rec, fm[req]); !found {
				return fmt.Errorf("record missing path %q for %q", fm[req], req)
			}
			ok = true
		}
		_ = ok
	}
	return nil
}

if err != nil { var m *importers.MappingError; if errors.As(err, &m) { log.Printf("fix mapping for target %s", m.Target) } }

When it happens

Trigger: FieldMap points to "startTime" but records use "started_at" (path mismatch); a dotted path like "meta.traceId" where the intermediate key differs or the nesting is absent on some records; one malformed record in the array missing the field entirely — mapping is per-record, so a single record without the path fails the whole import.

Common situations: Schema drift between export batches (vendor renamed a field); hand-written field map from documentation that is out of date; heterogeneous records where optional events lack the required timestamp field; case-sensitive path ('TraceId' vs 'traceId').

Related errors

• generic record %d: %w
• mapped source path %q for target %q was not found in any rec
• generic import requires a non-empty field map
• unknown target column %q (not a caveman.spans column)
• generic target %q requires a non-empty source path

AI-assisted analysis of JuliusBrussee/caveman@27d5a3981a (2026-08-15). Data as JSON: /api/errors/cda960b0daaf65b5. Report an issue: GitHub.