{"record":{"id":"b54ff6cae4c4e032","repo":"SigNoz/signoz","slug":"error-scanning-fallback-metadata-v","errorCode":null,"errorMessage":"error scanning fallback metadata: %v","messagePattern":"error scanning fallback metadata: (.+?)","errorType":"exception","errorClass":"model.ApiError","httpStatus":500,"severity":"error","filePath":"pkg/query-service/app/clickhouseReader/reader.go","lineNumber":5240,"sourceCode":"\t\t\tWHERE metric_name IN (%s)`, signozMetricDBName, signozTSTableNameV4, metricList)\n\t\t}\n\t\tvalueCtx := context.WithValue(ctx, \"clickhouse_max_threads\", constants.MetricsExplorerClickhouseThreads)\n\t\trows, err := r.db.Query(valueCtx, query)\n\t\tif err != nil {\n\t\t\treturn cachedMetadata, &model.ApiError{Typ: \"ClickhouseErr\", Err: fmt.Errorf(\"error querying time_series_v4 to get metrics metadata: %v\", err)}\n\t\t}\n\t\tdefer rows.Close()\n\t\tfor rows.Next() {\n\t\t\tmetadata := new(model.UpdateMetricsMetadata)\n\t\t\tif err := rows.Scan(\n\t\t\t\t&metadata.MetricName,\n\t\t\t\t&metadata.MetricType,\n\t\t\t\t&metadata.Description,\n\t\t\t\t&metadata.Temporality,\n\t\t\t\t&metadata.IsMonotonic,\n\t\t\t\t&metadata.Unit,\n\t\t\t); err != nil {\n\t\t\t\treturn cachedMetadata, &model.ApiError{Typ: \"ClickhouseErr\", Err: fmt.Errorf(\"error scanning fallback metadata: %v\", err)}\n\t\t\t}\n\n\t\t\tcacheKey := constants.UpdatedMetricsMetadataCachePrefix + metadata.MetricName\n\t\t\tif cacheErr := r.cache.Set(ctx, orgID, cacheKey, metadata, 0); cacheErr != nil {\n\t\t\t\tr.logger.Error(\"Failed to cache fallback metadata\", \"metric_name\", metadata.MetricName, errorsV2.Attr(cacheErr))\n\t\t\t}\n\t\t\tcachedMetadata[metadata.MetricName] = metadata\n\t\t}\n\t\tif rows.Err() != nil {\n\t\t\treturn cachedMetadata, &model.ApiError{Typ: \"ClickhouseErr\", Err: fmt.Errorf(\"error scanning fallback metadata: %v\", err)}\n\t\t}\n\t}\n\treturn cachedMetadata, nil\n}\n\nfunc (r *ClickHouseReader) SearchTraces(ctx context.Context, params *model.SearchTracesParams) (*[]model.SearchSpansResult, error) {\n\tctx = ctxtypes.NewContextWithCommentVals(ctx, map[string]string{\n\t\tinstrumentationtypes.TelemetrySignal:  telemetrytypes.SignalTraces.StringValue(),","sourceCodeStart":5222,"sourceCodeEnd":5258,"githubUrl":"https://github.com/SigNoz/signoz/blob/5069bf80b08f1f00d7e014eccc09902f9871004f/pkg/query-service/app/clickhouseReader/reader.go#L5222-L5258","documentation":"Returned when rows.Scan fails while reading a row of fallback metrics metadata from the time_series_v4 query result. The result columns do not match the scan targets (metric name, type, description, temporality, is_monotonic, unit) — usually a schema drift issue where the table has different column types/order than expected.","triggerScenarios":"GetMetricsMetadata fallback path scans a row whose columns mismatch the expected types: e.g. temporality or is_monotonic columns added/changed in a newer schema, NULL values in non-nullable scanned columns, or a ClickHouse version returning a type the driver cannot convert to the Go destination.","commonSituations":"SigNoz upgrade where time_series_v4 gained new columns but query-service is an older binary (or vice versa), partially applied migrations, or metrics without unit/description producing NULLs the scan cannot handle.","solutions":["Compare the actual column types via DESCRIBE TABLE signoz_metrics.time_series_v4 with the fields scanned (MetricName, MetricType, Description, Temporality, IsMonotonic, Unit)","Align query-service and ClickHouse schema versions by re-running SigNoz migrations so columns match","Wrap scanned columns with sql.Null* types or ensure the SELECT coalesces NULLs","Restart query-service after schema migration so prepared statements/statements cache are fresh"],"exampleFix":"// before\nif err := rows.Scan(&metadata.MetricName, &metadata.MetricType, &metadata.Description, &metadata.Temporality, &metadata.IsMonotonic, &metadata.Unit); err != nil {\n    return cachedMetadata, &model.ApiError{Typ: \"ClickhouseErr\", Err: fmt.Errorf(\"error scanning fallback metadata: %v\", err)}\n}\n\n// after (null-safe scan)\nvar unit, desc sql.NullString\nif err := rows.Scan(&metadata.MetricName, &metadata.MetricType, &desc, &metadata.Temporality, &metadata.IsMonotonic, &unit); err != nil { ... }\nif unit.Valid { metadata.Unit = unit.String }\nif desc.Valid { metadata.Description = desc.String }","handlingStrategy":"validation","validationCode":"// Verify column layout matches the scan before querying\nDESCRIBE TABLE signoz_metrics.time_series_v4\n// ensure temporality, is_monotonic, unit columns exist with expected types","typeGuard":null,"tryCatchPattern":"// On ClickhouseErr from the fallback path, degrade gracefully to cached metadata only\nif isClickhouseScanErr(err) {\n    log.Warn(\"fallback metadata scan failed; serving cache only\")\n    return cachedMetadata, nil\n}","preventionTips":["Use sql.Null* destinations for nullable metadata columns","Run schema migrations together with query-service upgrades","Add integration tests that scan all time_series_v4 columns"],"tags":["clickhouse","row-scan","schema-mismatch","metrics-metadata"],"backgroundTag":"row-scan-type-mismatch","analyzedSha":"5069bf80b08f1f00d7e014eccc09902f9871004f","analyzedAt":"2026-08-28T06:22:12.824Z","schemaVersion":2},"datasetVersion":"2026-08-28T11:17:15.048Z"}