openzipkin/zipkin · error · IllegalArgumentException

{annotationQueryString} query unsupported due to missing ann

Error message

{annotationQueryString} query unsupported due to missing annotation_query index

What it means

CassandraSpanStore.getTraces throws IllegalArgumentException ('{annotationQueryString} query unsupported due to missing annotation_query index') when a QueryRequest includes annotation/tag filters but the connected Cassandra schema lacks the annotation_query index table. Zipkin's Cassandra schema is versioned; older schemas (or custom keyspaces) without this table cannot serve annotation queries, so the request is rejected rather than silently returning wrong results.

Source

Thrown at zipkin-storage/cassandra/src/main/java/zipkin2/storage/cassandra/CassandraSpanStore.java:133

   * <p>The amount of backend calls increase in dimensions of query complexity, days of data, and
   * limit of traces requested. For example, a query like "http.path=/foo and error" will be two
   * select statements for the expression, possibly follow-up calls for pagination (when over 5K
   * rows match). Once IDs are parsed, there's one call for each 5K rows of span data. This means
   * "http.path=/foo and error" is minimally 3 network calls, the first two in parallel.
   */
  @Override public Call<List<List<Span>>> getTraces(QueryRequest request) {
    if (!searchEnabled) return Call.emptyList();

    TimestampRange timestampRange = timestampRange(request, indexTtl);
    // If we have to make multiple queries, over fetch on indexes as they don't return distinct
    // (trace id, timestamp) rows. This mitigates intersection resulting in < limit traces
    final int traceIndexFetchSize = request.limit() * indexFetchMultiplier;
    List<Call<Map<String, Long>>> callsToIntersect = new ArrayList<>();

    List<String> annotationKeys = CassandraUtil.annotationKeys(request);
    for (String annotationKey : annotationKeys) {
      if (spanTable == null) {
        throw new IllegalArgumentException(request.annotationQueryString()
          + " query unsupported due to missing annotation_query index");
      }
      callsToIntersect.add(
        spanTable.newCall(request.serviceName(), annotationKey, timestampRange, traceIndexFetchSize)
      );
    }

    // Bucketed calls can be expensive when service name isn't specified. This guards against abuse.
    if (request.remoteServiceName() != null
      || request.spanName() != null
      || request.minDuration() != null
      || callsToIntersect.isEmpty()) {
      callsToIntersect.add(newBucketedTraceIdCall(request, timestampRange, traceIndexFetchSize));
    }

    if (callsToIntersect.size() == 1) {
      return callsToIntersect.get(0)
        .map(traceIdsSortedByDescTimestamp())

View on GitHub (pinned to 878ce2a1fa)

Solutions

1. Install/upgrade the Cassandra schema to the current zipkin3 layout (let ensureSchema=true run against a fresh keyspace, or apply the schema upgrade scripts)
2. Point the server at a keyspace name matching the new schema (default zipkin3) instead of the legacy one
3. As a workaround, remove annotation/tag filters from the query so it uses supported indexes

Example fix

# before
zipkin.storage.type=cassandra
zipkin.storage.cassandra.keyspace=zipkin2   # legacy schema, no annotation_query

# after
zipkin.storage.type=cassandra
zipkin.storage.cassandra.keyspace=zipkin3
zipkin.storage.cassandra.ensure-schema=true

// before querying, check schema capability
boolean supportsAnnotationQuery = storage.spanStore() instanceof CassandraSpanStore;
// simplest: ensure keyspace was created with the current schema (zipkin3) and annotation_query exists
// cqlsh: DESCRIBE TABLE zipkin3.annotation_query;

catch (IllegalArgumentException e) if message contains 'missing annotation_query index' -> fall back to a query without annotation filters and surface a UI banner that the schema needs migration

When it happens

Trigger: QueryRequest with annotationQueryString (e.g. QueryRequest.newBuilder().serviceName(...).addAnnotation(...)) against a Cassandra keyspace created with an older schema or with ensureSchema against a pre-annotation-index layout where the spanTable factory is null.

Common situations: Upgrading Zipkin while reusing an old 'zipkin'/'zipkin2' keyspace that predates annotation_query; pointing at a keyspace installed by a much older release; environments where schema migration was never run.

Related errors

• remoteService={remoteService} unsupported due to missing tab
• key == null
• Start bucket ({startBucket}) > end bucket ({endBucket})
• Session initialization failed. See server logs
• schema not installed: apply %s, or set CASSANDRA_ENSURE_SCHE

AI-assisted analysis of openzipkin/zipkin@878ce2a1fa (2026-08-14). Data as JSON: /api/errors/6ae34ca947b56a9b. Report an issue: GitHub.