{"record":{"id":"b87aee25bea926eb","repo":"JuliusBrussee/caveman","slug":"generic-import-requires-a-q-mapping","errorCode":null,"errorMessage":"generic import requires a %q mapping","messagePattern":"generic import requires a %q mapping","errorType":"validation","errorClass":null,"httpStatus":null,"severity":"error","filePath":"shared/platform/importers/generic.go","lineNumber":104,"sourceCode":"\t}\n\treturn rows, nil\n}\n\nfunc validateGenericFieldMap(fieldMap map[string]string) error {\n\tif len(fieldMap) == 0 {\n\t\treturn fmt.Errorf(\"generic import requires a non-empty field map\")\n\t}\n\tfor target, path := range fieldMap {\n\t\tif _, ok := genericTargets[target]; !ok {\n\t\t\treturn fmt.Errorf(\"unknown target column %q (not a caveman.spans column)\", target)\n\t\t}\n\t\tif strings.TrimSpace(path) == \"\" {\n\t\t\treturn fmt.Errorf(\"generic target %q requires a non-empty source path\", target)\n\t\t}\n\t}\n\tfor _, target := range []string{\"trace_id\", \"span_id\", \"timestamp\"} {\n\t\tif _, ok := fieldMap[target]; !ok {\n\t\t\treturn fmt.Errorf(\"generic import requires a %q mapping\", target)\n\t\t}\n\t}\n\treturn nil\n}\n\nfunc decodeGenericRecords(data []byte) ([]map[string]any, error) {\n\t// Try a bare array first.\n\tvar arr []map[string]any\n\tif err := json.Unmarshal(data, &arr); err == nil {\n\t\treturn arr, nil\n\t}\n\t// Then a {\"data\":[...]} wrapper.\n\tvar env struct {\n\t\tData []map[string]any `json:\"data\"`\n\t}\n\tif err := json.Unmarshal(data, &env); err == nil && env.Data != nil {\n\t\treturn env.Data, nil\n\t}","sourceCodeStart":86,"sourceCodeEnd":122,"githubUrl":"https://github.com/JuliusBrussee/caveman/blob/27d5a3981a347890211bb1bf2439e5c821a63bc9/shared/platform/importers/generic.go#L86-L122","documentation":"The generic importer requires a field map that maps at least the three mandatory columns 'trace_id', 'span_id', and 'timestamp' to source JSON paths. validateGeneric loops over those required targets after checking each mapped target is a known caveman.spans column and returns this error when any of the three is absent. It is a fail-fast configuration check that happens before any data is decoded, so no rows are processed with an incomplete mapping.","triggerScenarios":"Calling generic import with opts.FieldMap that omits 'trace_id', 'span_id', or 'timestamp' (e.g. map[string]string{\"trace_id\": \"traceId\", \"name\": \"op\"} with no timestamp path). Also hit when the keys are misspelled ('traceId' instead of 'trace_id') since the check is an exact-key lookup in fieldMap.","commonSituations":"Building the FieldMap from user input or a config file where the timestamp mapping was considered optional; copy-pasting field-map YAML from a langfuse/helicone example that uses different target names; renaming targets assuming camelCase is accepted.","solutions":["Add mappings for all three required targets: fieldMap[\"trace_id\"], fieldMap[\"span_id\"], and fieldMap[\"timestamp\"] must each point to a non-empty source path","Check for misspellelling: targets must be the exact snake_case names trace_id/span_id/timestamp, not traceId/spanId/time","Confirm every other key in the map is a known caveman.spans column (unknown targets fail earlier with 'unknown target column')","If your data has no per-span timestamp, derive one before import instead of omitting the mapping"],"exampleFix":"// before\nopts := importers.Options{FieldMap: map[string]string{\n    \"trace_id\": \"traceId\",\n    \"span_id\":  \"spanId\",\n}}\n\n// after\nopts := importers.Options{FieldMap: map[string]string{\n    \"trace_id\":  \"traceId\",\n    \"span_id\":   \"spanId\",\n    \"timestamp\": \"startTime\",\n}}","handlingStrategy":"validation","validationCode":"func validateFieldMap(fm map[string]string) error {\n\tfor _, req := range []string{\"trace_id\", \"span_id\", \"timestamp\"} {\n\t\tif strings.TrimSpace(fm[req]) == \"\" {\n\t\t\treturn fmt.Errorf(\"field map missing required target %q\", req)\n\t\t}\n\t}\n\treturn nil\n}","typeGuard":"func isCompleteFieldMap(fm map[string]string) bool {\n\t_, t := fm[\"trace_id\"]\n\t_, s := fm[\"span_id\"]\n\t_, ts := fm[\"timestamp\"]\n\treturn t && s && ts\n}","tryCatchPattern":"if err := importers.Import(importers.FormatGeneric, data, opts); err != nil { if strings.Contains(err.Error(), \"requires a\") { /* fix FieldMap config */ } }","preventionTips":["Build FieldMap from a shared constant in tests and prod so required targets can't drift","Validate the field map in config loading, not at import time","Write a unit test asserting the three required targets are present"],"tags":["go","import","generic","validation"],"backgroundTag":null,"analyzedSha":"27d5a3981a347890211bb1bf2439e5c821a63bc9","analyzedAt":"2026-08-15T09:26:11.751Z","schemaVersion":2},"datasetVersion":"2026-08-15T22:17:37.221Z"}