{"record":{"id":"a9923737a5ac4679","repo":"apache/druid","slug":"cannot-operate-on-a-dimension-with-unknown-cardina-a99237","errorCode":null,"errorMessage":"Cannot operate on a dimension with unknown cardinality","messagePattern":"Cannot operate on a dimension with unknown cardinality","errorType":"exception","errorClass":"UnsupportedOperationException","httpStatus":null,"severity":"error","filePath":"processing/src/main/java/org/apache/druid/query/topn/types/StringTopNColumnAggregatesProcessor.java","lineNumber":67,"sourceCode":"    this.capabilities = capabilities;\n    this.dimensionValueConverter = DimensionHandlerUtils.converterFromTypeToType(ColumnType.STRING, dimensionType);\n  }\n\n  @Override\n  public int getCardinality(DimensionSelector selector)\n  {\n    // only report the underlying selector cardinality if the column the selector is for is dictionary encoded\n    if (capabilities.isDictionaryEncoded().isTrue()) {\n      return selector.getValueCardinality();\n    }\n    return DimensionDictionarySelector.CARDINALITY_UNKNOWN;\n  }\n\n  @Override\n  public Aggregator[][] getRowSelector(TopNQuery query, TopNParams params, TopNCursorInspector cursorInspector)\n  {\n    if (params.getCardinality() < 0) {\n      throw new UnsupportedOperationException(\"Cannot operate on a dimension with unknown cardinality\");\n    }\n\n    // This method is used for the HeapBasedTopNAlgorithm only.\n    // Unlike regular topN we cannot rely on ordering to optimize.\n    // Optimization possibly requires a reverse lookup from value to ID, which is\n    // not possible when applying an extraction function\n    final BaseTopNAlgorithm.AggregatorArrayProvider provider = new BaseTopNAlgorithm.AggregatorArrayProvider(\n        (DimensionSelector) params.getSelectorPlus().getSelector(),\n        query,\n        cursorInspector,\n        params.getCardinality()\n    );\n\n    return provider.build();\n  }\n\n  @Override\n  public void updateResults(TopNResultBuilder resultBuilder)","sourceCodeStart":49,"sourceCodeEnd":85,"githubUrl":"https://github.com/apache/druid/blob/9b90983fd291f26935af934383ce360473179e4d/processing/src/main/java/org/apache/druid/query/topn/types/StringTopNColumnAggregatesProcessor.java#L49-L85","documentation":"The string-dimension TopN aggregates processor (used by the heap-based TopN algorithm) needs a known dimension cardinality to allocate aggregator arrays. When the segment's dimension cardinality is unknown (negative, e.g. because an extraction function is applied to the dimension), the processor throws UnsupportedOperationException in getRowSelector.","triggerScenarios":"Running a topN query with a HeapBasedTopNAlgorithm against a dimension whose cardinality is unknown (< 0). This typically occurs when the dimension has an extraction function applied, or the column supplies unknown cardinality (e.g. certain dimension selectors during historical/realtime segment queries).","commonSituations":"TopN queries with dimensionExtractionFn on a column whose underlying cardinality cannot be computed; queries hitting realtime segments where cardinality is not yet fixed; tuning minTopNThreshold such that the heap-based algorithm is chosen for such queries.","solutions":["Remove the extraction function from the topN dimension (apply transformation after the query or via a virtual column/transform)","Increase minTopNThreshold context so a non-heap-based algorithm can be used, or restructure the query to avoid the heap-based path","Prefer groupBy over topN when dimension extraction makes cardinality unknown","Update the segment / ensure column cardinality is available (force rollup/republish if needed)"],"exampleFix":"// before\nDimensionSpec dim = new ExtractionDimensionSpec(\"col\", \"out\", ColumnType.STRING,\n    new CascadeExtractionFn(new ExtractionFn[] { new TimeFormatExtractionFn(...) }));\nTopNQuery q = new TopNQueryBuilder().dimension(dim).threshold(5000).build(); // heap-based path, unknown cardinality\n// after\nTopNQuery q = new TopNQueryBuilder().dimension(new DefaultDimensionSpec(\"col\", \"out\", ColumnType.STRING))\n    .threshold(1000).build(); // known cardinality, ordering-optimized path","handlingStrategy":"validation","validationCode":"DimensionSelector selector = params.getDimensionSelector();\nif (selector.getValueCardinality() < 0) {\n  throw new IllegalArgumentException(\"topN dimension has unknown cardinality; remove extractionFn or use groupBy\");\n}","typeGuard":"static boolean hasKnownCardinality(DimensionSelector selector) {\n  return selector != null && selector.getValueCardinality() >= 0;\n}","tryCatchPattern":"try {\n  Sequence<Result<TopNResultValue>> results = runner.run(queryPlus, ctx).toList();\n} catch (UnsupportedOperationException e) {\n  if (e.getMessage().contains(\"unknown cardinality\")) {\n    // fall back to a groupBy query or drop the extraction function\n  } else {\n    throw e;\n  }\n}","preventionTips":["Avoid extraction functions on topN dimensions; use transforms/virtual columns instead","Prefer groupBy for queries needing dimension transformation","Know that heap-based topN (large thresholds) requires known column cardinality","Test topN queries with dimension extraction on realtime segments before deploying"],"tags":["druid","topn","unsupported-operation"],"backgroundTag":"unsupported-operation","analyzedSha":"9b90983fd291f26935af934383ce360473179e4d","analyzedAt":"2026-09-07T13:32:30.957Z","contentChangedAt":"2026-09-07T13:32:30.957Z","schemaVersion":2},"datasetVersion":"2026-09-14T11:17:12.474Z"}