SigNoz/signoz · error · model.ApiError

error querying time_series_v4 to get metrics metadata: %v

Error message

error querying time_series_v4 to get metrics metadata: %v

What it means

Thrown by ClickHouseReader when the ClickHouse query against time_series_v4 (metrics metadata fallback) fails. This is a database-level failure: connection problems, syntax errors from malformed metric names, or a missing table/database in the deployed SigNoz schema. The error wraps the underlying ClickHouse driver error with %v.

Source

Thrown at pkg/query-service/app/clickhouseReader/reader.go:5227

		metricList := "'" + strings.Join(stillMissing, "', '") + "'"
		reductionEnabled := r.fl.BooleanOrEmpty(ctx, flagger.FeatureEnableMetricsReduction, featuretypes.NewFlaggerEvaluationContext(orgID))
		var query string
		if reductionEnabled {
			query = fmt.Sprintf(`SELECT DISTINCT metric_name, type, description, temporality, is_monotonic, unit
			FROM (
				SELECT metric_name, type, description, temporality, is_monotonic, unit FROM %s.%s WHERE metric_name IN (%s)
				UNION ALL
				SELECT metric_name, type, description, temporality, is_monotonic, unit FROM %s.%s WHERE metric_name IN (%s)
			)`, signozMetricDBName, signozTSTableNameV4, metricList, signozMetricDBName, signozTSTableNameV4Reduced, metricList)
		} else {
			query = fmt.Sprintf(`SELECT DISTINCT metric_name, type, description, temporality, is_monotonic, unit
			FROM %s.%s
			WHERE metric_name IN (%s)`, signozMetricDBName, signozTSTableNameV4, metricList)
		}
		valueCtx := context.WithValue(ctx, "clickhouse_max_threads", constants.MetricsExplorerClickhouseThreads)
		rows, err := r.db.Query(valueCtx, query)
		if err != nil {
			return cachedMetadata, &model.ApiError{Typ: "ClickhouseErr", Err: fmt.Errorf("error querying time_series_v4 to get metrics metadata: %v", err)}
		}
		defer rows.Close()
		for rows.Next() {
			metadata := new(model.UpdateMetricsMetadata)
			if err := rows.Scan(
				&metadata.MetricName,
				&metadata.MetricType,
				&metadata.Description,
				&metadata.Temporality,
				&metadata.IsMonotonic,
				&metadata.Unit,
			); err != nil {
				return cachedMetadata, &model.ApiError{Typ: "ClickhouseErr", Err: fmt.Errorf("error scanning fallback metadata: %v", err)}
			}

			cacheKey := constants.UpdatedMetricsMetadataCachePrefix + metadata.MetricName
			if cacheErr := r.cache.Set(ctx, orgID, cacheKey, metadata, 0); cacheErr != nil {
				r.logger.Error("Failed to cache fallback metadata", "metric_name", metadata.MetricName, errorsV2.Attr(cacheErr))

View on GitHub (pinned to 5069bf80b0)

Solutions

Verify ClickHouse is reachable and credentials in the query-service config are correct
Check that the database/table in signoz_metric_db_name (default signoz_metrics) and time_series_v4 exist: SHOW TABLES FROM signoz_metrics
Inspect query-service logs for the wrapped %v driver error to identify the exact ClickHouse failure
If the v4 table is missing, run SigNoz schema migrations or upgrade to a consistent version
Retry the metrics metadata request once ClickHouse is healthy

Example fix

// before
rows, err := r.db.Query(valueCtx, query)
if err != nil {
    return cachedMetadata, &model.ApiError{Typ: "ClickhouseErr", Err: fmt.Errorf("error querying time_series_v4 to get metrics metadata: %v", err)}
}

// after (include query context in the error for diagnosability)
rows, err := r.db.Query(valueCtx, query)
if err != nil {
    r.logger.Error("time_series_v4 metadata query failed", "query", query, errorsV2.Attr(err))
    return cachedMetadata, &model.ApiError{Typ: "ClickhouseErr", Err: fmt.Errorf("error querying time_series_v4 to get metrics metadata: %v", err)}
}

Defensive patterns

Strategy: retry

Validate before calling

// Pre-flight: verify ClickHouse reachable and table exists before requesting metadata
if err := pingClickhouse(); err != nil { log.Fatal("clickhouse unreachable: ", err) }
if _, err := ch.Query("SELECT 1 FROM signoz_metrics.time_series_v4 LIMIT 1"); err != nil { log.Fatal("time_series_v4 missing: ", err) }

Try / catch

// Treat ClickhouseErr as transient: backoff-retry up to 3 times, then surface to caller
var apiErr *model.ApiError
if errors.As(err, &apiErr) && apiErr.Typ == "ClickhouseErr" {
    if ok := retry(3, time.Second, func() error { _, e := getMetricsMetadata(ctx); return e }); !ok {
        return fmt.Errorf("metrics metadata unavailable: %w", err)
    }
}

Prevention

Monitor ClickHouse health and table existence with a readiness probe
Keep query-service and ClickHouse schema versions aligned via migrations
Cache metrics metadata to reduce repeated time_series_v4 queries

When it happens

Trigger: Calling GetMetricsMetadata (or an API that populates metrics metadata for the explorer) where r.db.Query on 'SELECT ... FROM signoz_metrics.time_series_v4 WHERE metric_name IN (...)' fails: ClickHouse is down/unreachable, signoz_metric_db_name or the v4 table does not exist (older SigNoz schema), or metricList is empty/malformed producing invalid SQL.

Common situations: ClickHouse connection misconfiguration (host/port/credentials in signoz config), upgrading SigNoz from a pre-v4 schema where time_series_v4 was not yet created, ClickHouse node restarted or out of resources, or a migration that renamed the metrics DB/table.

Related errors

AI-assisted analysis of SigNoz/signoz@5069bf80b0 (2026-08-28). Data as JSON: /api/errors/32452a0973ea38fd. Report an issue: GitHub.