jaegertracing/jaeger · critical

failed to query trace: %w

Error message

failed to query trace: %w

What it means

Reader.GetTraces yields this error when the clickhouse-go conn.Query call for a single trace ID fails. This happens before any rows are read, so it reflects query construction, connection, or server-side rejection of the SELECT built by buildGetTracesQuery. The iterator stops immediately and surfaces the underlying driver error wrapped with %w.

Source

Thrown at internal/storage/v2/clickhouse/tracestore/reader.go:79

func (*Reader) SearchCapabilities(context.Context) (tracestore.SearchCapabilities, error) {
	return tracestore.SearchCapabilities{
		// The search SQL starts from "WHERE 1=1" and appends the service predicate only
		// when the query carries a name, so an omitted name matches spans from every
		// service.
		WithoutServiceName: true,
	}, nil
}

func (r *Reader) GetTraces(
	ctx context.Context,
	traceIDs ...tracestore.GetTraceParams,
) iter.Seq2[[]ptrace.Traces, error] {
	return func(yield func([]ptrace.Traces, error) bool) {
		for _, traceID := range traceIDs {
			query, args := buildGetTracesQuery(traceID)
			rows, err := r.conn.Query(ctx, query, args...)
			if err != nil {
				yield(nil, fmt.Errorf("failed to query trace: %w", err))
				return
			}

			var errs []error
			for rows.Next() {
				span, scanErr := dbmodel.ScanRow(rows)
				if scanErr != nil {
					errs = append(errs, fmt.Errorf("failed to scan span row: %w", scanErr))
					break
				}
				trace := dbmodel.FromRow(span)
				if !yield([]ptrace.Traces{trace}, nil) {
					_ = rows.Close()
					return
				}
			}
			if rowsErr := rows.Err(); rowsErr != nil {
				errs = append(errs, fmt.Errorf("failed to read span rows: %w", rowsErr))

View on GitHub (pinned to 806f444784)

Solutions

1. Verify ClickHouse connectivity (host, port, credentials, TLS) using the configured DSN, e.g. with clickhouse-client
2. Confirm the spans table exists and migrations were applied
3. Inspect the wrapped driver error (errors.Unwrap / %v of the yielded error) for the precise server message
4. Add retry/backoff around GetTraces for transient network failures

Example fix

// before (no retry, hard failure on transient error)
traces, err := reader.GetTraces(ctx, ids)
// after
traces, err := retry.Do(ctx, func() (iter.Seq2[[]ptrace.Traces, error], error) {
	return reader.GetTraces(ctx, ids)
}, retry.Attempts(3))

// preflight connectivity before querying
conn := clickhouse.Open(&clickhouse.Options{Addr: []string{addr}})
if err := conn.Ping(ctx); err != nil {
	return fmt.Errorf("clickhouse unreachable: %w", err)
}

for traces, err := range reader.GetTraces(ctx, ids) {
	if err != nil {
		if strings.Contains(err.Error(), "failed to query trace") {
			// transient driver failure — retry with backoff
			continue
		}
		return err
	}
	process(traces)
}

When it happens

Trigger: Calling GetTraces with a trace ID while ClickHouse is unreachable, the connection pool is exhausted, credentials are wrong, the database/table is missing, or the generated SQL is rejected by the server (syntax/permissions).

Common situations: ClickHouse pod restarted or network partition in k8s; misconfigured DSN (wrong host/port/database) in the storage config; schema migrations not applied so the spans table doesn't exist; TLS/auth mismatch between client and server.

Related errors

• failed to query services: %w
• failed to query trace IDs: %w
• failed to send batch: %w
• failed executing metrics query: %w
• failed executing metrics query: %w

AI-assisted analysis of jaegertracing/jaeger@806f444784 (2026-09-01). Data as JSON: /api/errors/11e671eb7cd0b1c6. Report an issue: GitHub.