JuliusBrussee/caveman · error

generic import requires a %q mapping

Error message

generic import requires a %q mapping

What it means

The generic importer requires a field map that maps at least the three mandatory columns 'trace_id', 'span_id', and 'timestamp' to source JSON paths. validateGeneric loops over those required targets after checking each mapped target is a known caveman.spans column and returns this error when any of the three is absent. It is a fail-fast configuration check that happens before any data is decoded, so no rows are processed with an incomplete mapping.

Source

Thrown at shared/platform/importers/generic.go:104

	}
	return rows, nil
}

func validateGenericFieldMap(fieldMap map[string]string) error {
	if len(fieldMap) == 0 {
		return fmt.Errorf("generic import requires a non-empty field map")
	}
	for target, path := range fieldMap {
		if _, ok := genericTargets[target]; !ok {
			return fmt.Errorf("unknown target column %q (not a caveman.spans column)", target)
		}
		if strings.TrimSpace(path) == "" {
			return fmt.Errorf("generic target %q requires a non-empty source path", target)
		}
	}
	for _, target := range []string{"trace_id", "span_id", "timestamp"} {
		if _, ok := fieldMap[target]; !ok {
			return fmt.Errorf("generic import requires a %q mapping", target)
		}
	}
	return nil
}

func decodeGenericRecords(data []byte) ([]map[string]any, error) {
	// Try a bare array first.
	var arr []map[string]any
	if err := json.Unmarshal(data, &arr); err == nil {
		return arr, nil
	}
	// Then a {"data":[...]} wrapper.
	var env struct {
		Data []map[string]any `json:"data"`
	}
	if err := json.Unmarshal(data, &env); err == nil && env.Data != nil {
		return env.Data, nil
	}

View on GitHub (pinned to 27d5a3981a)

Solutions

1. Add mappings for all three required targets: fieldMap["trace_id"], fieldMap["span_id"], and fieldMap["timestamp"] must each point to a non-empty source path
2. Check for misspellelling: targets must be the exact snake_case names trace_id/span_id/timestamp, not traceId/spanId/time
3. Confirm every other key in the map is a known caveman.spans column (unknown targets fail earlier with 'unknown target column')
4. If your data has no per-span timestamp, derive one before import instead of omitting the mapping

Example fix

// before
opts := importers.Options{FieldMap: map[string]string{
    "trace_id": "traceId",
    "span_id":  "spanId",
}}

// after
opts := importers.Options{FieldMap: map[string]string{
    "trace_id":  "traceId",
    "span_id":   "spanId",
    "timestamp": "startTime",
}}

func validateFieldMap(fm map[string]string) error {
	for _, req := range []string{"trace_id", "span_id", "timestamp"} {
		if strings.TrimSpace(fm[req]) == "" {
			return fmt.Errorf("field map missing required target %q", req)
		}
	}
	return nil
}

func isCompleteFieldMap(fm map[string]string) bool {
	_, t := fm["trace_id"]
	_, s := fm["span_id"]
	_, ts := fm["timestamp"]
	return t && s && ts
}

if err := importers.Import(importers.FormatGeneric, data, opts); err != nil { if strings.Contains(err.Error(), "requires a") { /* fix FieldMap config */ } }

When it happens

Trigger: Calling generic import with opts.FieldMap that omits 'trace_id', 'span_id', or 'timestamp' (e.g. map[string]string{"trace_id": "traceId", "name": "op"} with no timestamp path). Also hit when the keys are misspelled ('traceId' instead of 'trace_id') since the check is an exact-key lookup in fieldMap.

Common situations: Building the FieldMap from user input or a config file where the timestamp mapping was considered optional; copy-pasting field-map YAML from a langfuse/helicone example that uses different target names; renaming targets assuming camelCase is accepted.

Related errors

• unknown target column %q (not a caveman.spans column)
• generic target %q requires a non-empty source path
• generic decode: expected a JSON array or {"data":[...]} of o
• mapped source path %q for required target %q was not found
• mapped trace_id is empty

AI-assisted analysis of JuliusBrussee/caveman@27d5a3981a (2026-08-15). Data as JSON: /api/errors/b87aee25bea926eb. Report an issue: GitHub.