{"record":{"id":"6152301cbdb0051b","repo":"jaegertracing/jaeger","slug":"failed-to-scan-span-row-w","errorCode":null,"errorMessage":"failed to scan span row: %w","messagePattern":"failed to scan span row: %w","errorType":"exception","errorClass":null,"httpStatus":null,"severity":"error","filePath":"internal/storage/v2/clickhouse/tracestore/reader.go","lineNumber":87,"sourceCode":"\nfunc (r *Reader) GetTraces(\n\tctx context.Context,\n\ttraceIDs ...tracestore.GetTraceParams,\n) iter.Seq2[[]ptrace.Traces, error] {\n\treturn func(yield func([]ptrace.Traces, error) bool) {\n\t\tfor _, traceID := range traceIDs {\n\t\t\tquery, args := buildGetTracesQuery(traceID)\n\t\t\trows, err := r.conn.Query(ctx, query, args...)\n\t\t\tif err != nil {\n\t\t\t\tyield(nil, fmt.Errorf(\"failed to query trace: %w\", err))\n\t\t\t\treturn\n\t\t\t}\n\n\t\t\tvar errs []error\n\t\t\tfor rows.Next() {\n\t\t\t\tspan, scanErr := dbmodel.ScanRow(rows)\n\t\t\t\tif scanErr != nil {\n\t\t\t\t\terrs = append(errs, fmt.Errorf(\"failed to scan span row: %w\", scanErr))\n\t\t\t\t\tbreak\n\t\t\t\t}\n\t\t\t\ttrace := dbmodel.FromRow(span)\n\t\t\t\tif !yield([]ptrace.Traces{trace}, nil) {\n\t\t\t\t\t_ = rows.Close()\n\t\t\t\t\treturn\n\t\t\t\t}\n\t\t\t}\n\t\t\tif rowsErr := rows.Err(); rowsErr != nil {\n\t\t\t\terrs = append(errs, fmt.Errorf(\"failed to read span rows: %w\", rowsErr))\n\t\t\t}\n\t\t\tif closeErr := rows.Close(); closeErr != nil {\n\t\t\t\terrs = append(errs, fmt.Errorf(\"failed to close rows: %w\", closeErr))\n\t\t\t}\n\t\t\tif err := errors.Join(errs...); err != nil {\n\t\t\t\tyield(nil, err)\n\t\t\t\treturn\n\t\t\t}","sourceCodeStart":69,"sourceCodeEnd":105,"githubUrl":"https://github.com/jaegertracing/jaeger/blob/806f4447841ecdb60519f408b004a599d515f437/internal/storage/v2/clickhouse/tracestore/reader.go#L69-L105","documentation":"After the query succeeds, each row is decoded via dbmodel.ScanRow; if row scanning fails (column/type mismatch between the ClickHouse schema and the dbmodel scan targets), the error is appended and row iteration breaks. The wrapped error is the clickhouse driver's scan error identifying which column/type failed.","triggerScenarios":"Calling GetTraces against a table whose schema differs from what dbmodel.ScanRow expects — e.g. columns added/removed/renamed by a schema version mismatch, NULL values in non-nullable scan targets, or incompatible column types (String vs UInt64).","commonSituations":"Running new jaeger code against an old un-migrated ClickHouse schema (or vice versa); manual schema edits; data written by a different jaeger version with different column encoding.","solutions":["Apply the matching ClickHouse schema migrations for your jaeger version","Compare the table's DESCRIBE output with the columns dbmodel.ScanRow expects and reconcile differences","Find the failing column in the wrapped error and fix its type/nullability","Re-ingest or transform rows written by incompatible producer versions"],"exampleFix":"// before (old schema missing a column ScanRow expects)\nALTER TABLE spans ADD COLUMN events String;\n// after (run provided migrations instead of ad-hoc edits)\njaeger-ch-schema migrate --version=<matching-version>","handlingStrategy":"try-catch","validationCode":"// verify schema compatibility before deploying\ncols, err := conn.Query(ctx, \"DESCRIBE TABLE spans\")\n// compare column names/types/nullability against dbmodel.ScanRow's expectations","typeGuard":null,"tryCatchPattern":"for traces, err := range reader.GetTraces(ctx, ids) {\n\tif err != nil {\n\t\tif strings.Contains(err.Error(), \"failed to scan span row\") {\n\t\t\tlog.Printf(\"schema mismatch on span row: %v\", err) // investigate schema, not network\n\t\t}\n\t\treturn err\n\t}\n\tprocess(traces)\n}","preventionTips":["Always run the jaeger ClickHouse schema migrations matching your binary version","Add a startup schema-version check against the migrations table","Never hand-edit ClickHouse table schemas for jaeger tables","Test upgrades against a copy of production data to catch scan mismatches"],"tags":["clickhouse","schema","deserialization","scan"],"backgroundTag":"row-scan-type-mismatch","analyzedSha":"806f4447841ecdb60519f408b004a599d515f437","analyzedAt":"2026-09-01T02:39:22.140Z","contentChangedAt":null,"schemaVersion":2},"datasetVersion":"2026-09-08T05:18:18.240Z"}