openzipkin/zipkin · error · IllegalArgumentException

remoteService={remoteService} unsupported due to missing tab

Error message

remoteService={remoteService} unsupported due to missing table remote_service_by_service

What it means

CassandraSpanStore.newBucketedTraceIdCall throws IllegalArgumentException ('remoteService={remoteService} unsupported due to missing table remote_service_by_service') when a trace search filters by remote service name but the Cassandra schema does not have the remote_service_by_service table. That table was added in a later schema version; without it, remote-service filtering cannot be answered correctly and the query is rejected.

Source

Thrown at zipkin-storage/cassandra/src/main/java/zipkin2/storage/cassandra/CassandraSpanStore.java:199

    int endBucket = durationIndexBucket(timestampRange.endMillis * 1000);
    if (startBucket > endBucket) {
      throw new IllegalArgumentException(
        "Start bucket (" + startBucket + ") > end bucket (" + endBucket + ")");
    }

    // "" isn't a real value. it is used to template bucketed calls and replaced later
    String serviceName = null != request.serviceName() ? request.serviceName() : "";

    // TODO: ideally, the buckets are traversed backwards, only spawning queries for older buckets
    // if younger buckets are empty. This will be an async continuation, punted for now.
    List<SelectTraceIdsFromServiceSpan.Input> serviceSpans = new ArrayList<>();
    List<SelectTraceIdsFromServiceRemoteService.Input> serviceRemoteServices = new ArrayList<>();
    String remoteService = request.remoteServiceName();
    for (int bucket = endBucket; bucket >= startBucket; bucket--) {
      boolean addSpanQuery = true;
      if (remoteService != null) {
        if (traceIdsFromServiceRemoteService == null) {
          throw new IllegalArgumentException("remoteService=" + remoteService
            + " unsupported due to missing table " + TABLE_SERVICE_REMOTE_SERVICES);
        }
        serviceRemoteServices.add(
          traceIdsFromServiceRemoteService.newInput(
            serviceName,
            remoteService,
            bucket,
            timestampRange,
            traceIndexFetchSize));
        // If the remote service query can satisfy the request, don't make a redundant span query
        addSpanQuery = !spanName.isEmpty() || minDuration != null;
      }
      if (!addSpanQuery) continue;

      serviceSpans.add(
        traceIdsFromServiceSpan.newInput(
          serviceName,
          spanName,

View on GitHub (pinned to 878ce2a1fa)

Solutions

  1. Migrate to the current Cassandra schema (fresh keyspace zipkin3 or apply upgrade scripts) so remote_service_by_service exists
  2. Set zipkin.storage.cassandra.keyspace to a keyspace created with the current schema
  3. Until migrated, avoid remoteServiceName filters in queries

Example fix

# before
zipkin.storage.cassandra.keyspace=zipkin2  # predates remote_service_by_service

# after
zipkin.storage.cassandra.keyspace=zipkin3
zipkin.storage.cassandra.ensure-schema=true
Defensive patterns

Strategy: validation

Validate before calling

// cqlsh check before enabling remote-service filters
// DESCRIBE TABLE zipkin3.remote_service_by_service;  -- must succeed

Try / catch

catch (IllegalArgumentException e) if message contains 'remote_service_by_service' -> retry the query without remoteServiceName and flag the schema as needing migration

Prevention

When it happens

Trigger: QueryRequest.newBuilder().serviceName(...).remoteServiceName(...) executed against a keyspace on the older schema where the traceIdsFromServiceRemoteService factory (indexTtl-based table) is null.

Common situations: Reusing a legacy zipkin2 keyspace after upgrading the Zipkin server; querying with the Lens UI's remote-service filter against an unmigrated Cassandra cluster.

Related errors


AI-assisted analysis of openzipkin/zipkin@878ce2a1fa (2026-08-14). Data as JSON: /api/errors/f432e99baf3de696. Report an issue: GitHub.