jaegertracing/jaeger · error
error reading operation_names from storage: %w
Error message
error reading operation_names from storage: %w
What it means
getOperationsV1 scans all rows from the legacy operation_names table via gocql's iterator; if iter.Close() returns an error (query failed mid-scan, timeout, node down), the raw driver error is wrapped with this message to give context about which read failed.
Source
Thrown at internal/storage/v1/cassandra/spanstore/operation_names.go:174
err := query.Exec()
return err == nil
}
func getOperationsV1(
s *OperationNamesStorage,
query tracestore.OperationQueryParams,
) ([]tracestore.Operation, error) {
iter := s.session.Query(s.table.queryStmt, query.ServiceName).Iter()
var operation string
var operations []tracestore.Operation
for iter.Scan(&operation) {
operations = append(operations, tracestore.Operation{
Name: operation,
})
}
if err := iter.Close(); err != nil {
err = fmt.Errorf("error reading operation_names from storage: %w", err)
return nil, err
}
return operations, nil
}
func getOperationsV2(
s *OperationNamesStorage,
query tracestore.OperationQueryParams,
) ([]tracestore.Operation, error) {
var casQuery cassandra.Query
if query.SpanKind == "" {
// Get operations for all spanKind
casQuery = s.session.Query(s.table.queryStmt, query.ServiceName)
} else {
// Get operations for given spanKind
casQuery = s.session.Query(s.table.queryByKindStmt, query.ServiceName, query.SpanKind)
}View on GitHub (pinned to 806f444784)
Solutions
- Check Cassandra logs / system.traces for the underlying read failure (timeouts, unavailable) and address cluster health.
- Upgrade the schema to operation_names_v2 (run latest schema script), which is bucketed and less timeout-prone.
- Retry the GetServices/GetOperations call once the cluster recovers; it is a read-only operation.
- Tune Cassandra client timeouts/retry policy in the Jaeger storage configuration if timeouts are frequent.
Defensive patterns
Strategy: retry
Try / catch
ops, err := reader.GetOperations(ctx, tracestore.OperationQuery{ServiceName: svc})
if err != nil {
if strings.Contains(err.Error(), "error reading operation_names") {
// transient Cassandra read failure: backoff and retry
}
return err
} Prevention
- Monitor Cassandra read timeout metrics and alert on spikes.
- Migrate to operation_names_v2 (bucketed) schema for less timeout-prone reads.
- Size the cluster for the service/operation cardinality.
- Use conservative client timeouts with a retry-enabled gocql policy.
When it happens
Trigger: Calling GetOperations (SpanReader) against a keyspace on schema version 1 (operation_names table) while the underlying Cassandra query fails — coordinator timeout, node unavailability, tombstone/gc_grace issues during a full-scan query.
Common situations: Cassandra cluster under load causing read timeouts on the unbucketed v1 table; network blips between Jaeger and Cassandra; large per-service operation lists hitting read timeouts.
Related errors
- error reading %s from storage: %w
- error reading traces from storage: %w
- error reading service_names from storage: %w
- invalid version
- failed to Exec query '%s': %w
AI-assisted analysis of jaegertracing/jaeger@806f444784 (2026-09-01).
Data as JSON: /api/errors/989dc194378433db.
Report an issue: GitHub.