apache/druid · error · UnsupportedOperationException

Cannot operate on a dimension with unknown cardinality

Error message

Cannot operate on a dimension with unknown cardinality

What it means

The string-dimension TopN aggregates processor (used by the heap-based TopN algorithm) needs a known dimension cardinality to allocate aggregator arrays. When the segment's dimension cardinality is unknown (negative, e.g. because an extraction function is applied to the dimension), the processor throws UnsupportedOperationException in getRowSelector.

Source

Thrown at processing/src/main/java/org/apache/druid/query/topn/types/StringTopNColumnAggregatesProcessor.java:67

    this.capabilities = capabilities;
    this.dimensionValueConverter = DimensionHandlerUtils.converterFromTypeToType(ColumnType.STRING, dimensionType);
  }

  @Override
  public int getCardinality(DimensionSelector selector)
  {
    // only report the underlying selector cardinality if the column the selector is for is dictionary encoded
    if (capabilities.isDictionaryEncoded().isTrue()) {
      return selector.getValueCardinality();
    }
    return DimensionDictionarySelector.CARDINALITY_UNKNOWN;
  }

  @Override
  public Aggregator[][] getRowSelector(TopNQuery query, TopNParams params, TopNCursorInspector cursorInspector)
  {
    if (params.getCardinality() < 0) {
      throw new UnsupportedOperationException("Cannot operate on a dimension with unknown cardinality");
    }

    // This method is used for the HeapBasedTopNAlgorithm only.
    // Unlike regular topN we cannot rely on ordering to optimize.
    // Optimization possibly requires a reverse lookup from value to ID, which is
    // not possible when applying an extraction function
    final BaseTopNAlgorithm.AggregatorArrayProvider provider = new BaseTopNAlgorithm.AggregatorArrayProvider(
        (DimensionSelector) params.getSelectorPlus().getSelector(),
        query,
        cursorInspector,
        params.getCardinality()
    );

    return provider.build();
  }

  @Override
  public void updateResults(TopNResultBuilder resultBuilder)

View on GitHub (pinned to 9b90983fd2)

Solutions

  1. Remove the extraction function from the topN dimension (apply transformation after the query or via a virtual column/transform)
  2. Increase minTopNThreshold context so a non-heap-based algorithm can be used, or restructure the query to avoid the heap-based path
  3. Prefer groupBy over topN when dimension extraction makes cardinality unknown
  4. Update the segment / ensure column cardinality is available (force rollup/republish if needed)

Example fix

// before
DimensionSpec dim = new ExtractionDimensionSpec("col", "out", ColumnType.STRING,
    new CascadeExtractionFn(new ExtractionFn[] { new TimeFormatExtractionFn(...) }));
TopNQuery q = new TopNQueryBuilder().dimension(dim).threshold(5000).build(); // heap-based path, unknown cardinality
// after
TopNQuery q = new TopNQueryBuilder().dimension(new DefaultDimensionSpec("col", "out", ColumnType.STRING))
    .threshold(1000).build(); // known cardinality, ordering-optimized path
Defensive patterns

Strategy: validation

Validate before calling

DimensionSelector selector = params.getDimensionSelector();
if (selector.getValueCardinality() < 0) {
  throw new IllegalArgumentException("topN dimension has unknown cardinality; remove extractionFn or use groupBy");
}

Type guard

static boolean hasKnownCardinality(DimensionSelector selector) {
  return selector != null && selector.getValueCardinality() >= 0;
}

Try / catch

try {
  Sequence<Result<TopNResultValue>> results = runner.run(queryPlus, ctx).toList();
} catch (UnsupportedOperationException e) {
  if (e.getMessage().contains("unknown cardinality")) {
    // fall back to a groupBy query or drop the extraction function
  } else {
    throw e;
  }
}

Prevention

When it happens

Trigger: Running a topN query with a HeapBasedTopNAlgorithm against a dimension whose cardinality is unknown (< 0). This typically occurs when the dimension has an extraction function applied, or the column supplies unknown cardinality (e.g. certain dimension selectors during historical/realtime segment queries).

Common situations: TopN queries with dimensionExtractionFn on a column whose underlying cardinality cannot be computed; queries hitting realtime segments where cardinality is not yet fixed; tuning minTopNThreshold such that the heap-based algorithm is chosen for such queries.

Understand the failure class

Background: UnsupportedOperationException and "is not supported" errors: when a library deliberately refuses a call — this error's family across 30 libraries.

Related errors

AI-assisted analysis of apache/druid@9b90983fd2 (2026-09-07). Data as JSON: /api/errors/a9923737a5ac4679. Report an issue: GitHub.