{"record":{"id":"965f48466f42a546","repo":"jaegertracing/jaeger","slug":"failed-to-query-services-w","errorCode":null,"errorMessage":"failed to query services: %w","messagePattern":"failed to query services: %w","errorType":"exception","errorClass":null,"httpStatus":null,"severity":"critical","filePath":"internal/storage/v2/clickhouse/tracestore/reader.go","lineNumber":113,"sourceCode":"\t\t\t}\n\t\t\tif rowsErr := rows.Err(); rowsErr != nil {\n\t\t\t\terrs = append(errs, fmt.Errorf(\"failed to read span rows: %w\", rowsErr))\n\t\t\t}\n\t\t\tif closeErr := rows.Close(); closeErr != nil {\n\t\t\t\terrs = append(errs, fmt.Errorf(\"failed to close rows: %w\", closeErr))\n\t\t\t}\n\t\t\tif err := errors.Join(errs...); err != nil {\n\t\t\t\tyield(nil, err)\n\t\t\t\treturn\n\t\t\t}\n\t\t}\n\t}\n}\n\nfunc (r *Reader) GetServices(ctx context.Context) ([]string, error) {\n\trows, err := r.conn.Query(ctx, sql.SelectServices)\n\tif err != nil {\n\t\treturn nil, fmt.Errorf(\"failed to query services: %w\", err)\n\t}\n\n\tvar (\n\t\tservices []string\n\t\terrs     []error\n\t)\n\tfor rows.Next() {\n\t\tvar service dbmodel.Service\n\t\tif scanErr := rows.ScanStruct(&service); scanErr != nil {\n\t\t\terrs = append(errs, fmt.Errorf(\"failed to scan row: %w\", scanErr))\n\t\t\tbreak\n\t\t}\n\t\tservices = append(services, service.Name)\n\t}\n\tif rowsErr := rows.Err(); rowsErr != nil {\n\t\terrs = append(errs, fmt.Errorf(\"failed to read service rows: %w\", rowsErr))\n\t}\n\tif closeErr := rows.Close(); closeErr != nil {","sourceCodeStart":95,"sourceCodeEnd":131,"githubUrl":"https://github.com/jaegertracing/jaeger/blob/806f4447841ecdb60519f408b004a599d515f437/internal/storage/v2/clickhouse/tracestore/reader.go#L95-L131","documentation":"Reader.GetServices returns this when the conn.Query call executing sql.SelectServices fails before any rows are read. It wraps the clickhouse-go driver error, so the root cause is at the connection, authentication, or server-rejection layer, not in result decoding.","triggerScenarios":"Calling GetServices while ClickHouse is down/unreachable, credentials are invalid, the service_names table (or whatever sql.SelectServices targets) is missing, or the user lacks SELECT permission.","commonSituations":"Fresh deployment where schema/migrations weren't applied; wrong DSN in jaeger storage config; ClickHouse user lacking grants on the target table; DNS/network issues between jaeger and the ClickHouse cluster.","solutions":["Run the sql.SelectServices query manually with the same credentials to see the server's error","Verify DSN configuration (host, port, database, auth) in the jaeger storage config","Apply schema migrations so the services table exists","GRANT SELECT on the required tables to the jaeger ClickHouse user"],"exampleFix":"// before\nGRANT SELECT ON default.traces TO jaeger_user; -- services table not granted\n// after\nGRANT SELECT ON default.* TO jaeger_user; -- or grant the specific services table","handlingStrategy":"retry","validationCode":"// preflight services query availability\nif err := conn.Ping(ctx); err != nil {\n\treturn fmt.Errorf(\"clickhouse unreachable: %w\", err)\n}\n// and confirm the target table exists\nvar exists uint8\nconn.QueryRow(ctx, \"EXISTS TABLE service_names\").Scan(&exists)","typeGuard":null,"tryCatchPattern":"services, err := reader.GetServices(ctx)\nif err != nil {\n\tif strings.Contains(err.Error(), \"failed to query services\") {\n\t\treturn cachedServices, nil // serve stale list while ClickHouse recovers\n\t}\n\treturn nil, err\n}","preventionTips":["Run schema migrations as a deployment prerequisite","GRANT SELECT on all jaeger tables to the configured user","Validate the DSN with clickhouse-client before deploying","Cache the service list to tolerate transient query failures"],"tags":["clickhouse","database","query","services"],"backgroundTag":"clickhouse-query-failed","analyzedSha":"806f4447841ecdb60519f408b004a599d515f437","analyzedAt":"2026-09-01T02:39:22.140Z","contentChangedAt":null,"schemaVersion":2},"datasetVersion":"2026-09-08T05:18:18.240Z"}