What causes "error in processing sql query" in SigNoz/signoz?

Check query-service logs/traces at the same timestamp for the underlying CH error

error in processing sql query

Error message

error in processing sql query

What it means

Error returned by GetServicesList when the ClickHouse query for distinct service names over the last day fails. The underlying err is discarded (not wrapped), so the message is generic; the real cause appears only in surrounding logs/traces.

Source

Thrown at pkg/query-service/app/clickhouseReader/reader.go:283

	if query.Stats != "" {
		qs = stats.NewQueryStats(qry.Stats())
	}

	qry.Close()
	return res, &qs, nil
}

func (r *ClickHouseReader) GetServicesList(ctx context.Context) (*[]string, error) {
	ctx = ctxtypes.NewContextWithCommentVals(ctx, map[string]string{
		instrumentationtypes.TelemetrySignal:  telemetrytypes.SignalTraces.StringValue(),
		instrumentationtypes.CodeNamespace:    "clickhouse-reader",
		instrumentationtypes.CodeFunctionName: "GetServicesList",
	})

	services := []string{}
	rows, err := r.db.Query(ctx, fmt.Sprintf(`SELECT DISTINCT resource_string_service$$name FROM %s.%s WHERE ts_bucket_start > (toUnixTimestamp(now() - INTERVAL 1 DAY) - 1800) AND toDate(timestamp) > now() - INTERVAL 1 DAY`, r.TraceDB, r.traceTableName))
	if err != nil {
		return nil, fmt.Errorf("error in processing sql query")
	}

	defer rows.Close()
	for rows.Next() {
		var serviceName string
		if err := rows.Scan(&serviceName); err != nil {
			return &services, err
		}
		services = append(services, serviceName)
	}

	return &services, nil
}

func (r *ClickHouseReader) GetTopLevelOperations(ctx context.Context, start, end time.Time, services []string) (*map[string][]string, *model.ApiError) {
	ctx = ctxtypes.NewContextWithCommentVals(ctx, map[string]string{
		instrumentationtypes.TelemetrySignal:  telemetrytypes.SignalTraces.StringValue(),
		instrumentationtypes.CodeNamespace:    "clickhouse-reader",

View on GitHub (pinned to 5069bf80b0)

Solutions

Check query-service logs/traces at the same timestamp for the underlying CH error
Verify CLICKHOUSE_URL/trace DB and table config match the actual ClickHouse schema
Confirm traces tables exist and migrations ran
Fix ClickHouse health/resources, then retry

Defensive patterns

Strategy: try-catch

Validate before calling

// Pre-flight CH and table checks
if !chHealthy(ctx) || !tableExists(ctx, ch, r.TraceDB, r.traceTableName) {
    return fmt.Errorf("traces store unavailable")
}

Type guard

func isServicesListErr(err error) bool {
    return err != nil && strings.Contains(err.Error(), "error in processing sql query")
}

Try / catch

svc, err := reader.GetServicesList(ctx)
if isServicesListErr(err) {
    logChDiagnostics() // real cause only in logs
    return []string{}, err
}

Prevention

Monitor ClickHouse availability from query-service
Validate TraceDB/traceTableName env config at startup
Complete CH migrations before serving traffic

When it happens

Trigger: Calling getServicesList / the services list API when r.db.Query against the traces table (SIGNOZ_TRACES_DB.<traces table>) fails: bad connection, missing table, syntax issues after schema drift.

Common situations: ClickHouse down or misconfigured (wrong TraceDB/table name from env), cold start where traces tables don't exist yet, upgrade where traceTableName changed, or CH overload.

Related errors

AI-assisted analysis of SigNoz/signoz@5069bf80b0 (2026-08-28). Data as JSON: /api/errors/469199c2b1f5047c. Report an issue: GitHub.