{"record":{"id":"1f12c2d5b0479f83","repo":"apache/druid","slug":"got-a-join-with-a-cartesian-product-that-exceeds","errorCode":null,"errorMessage":"Got a join, with a cartesian product that exceeds 1,000,000 rows, cannot handle it","messagePattern":"Got a join, with a cartesian product that exceeds 1,000,000 rows, cannot handle it","errorType":"exception","errorClass":"IllegalArgumentException","httpStatus":null,"severity":"error","filePath":"processing/src/main/java/org/apache/druid/query/operator/join/SortedInnerJoinOperator.java","lineNumber":357,"sourceCode":"        if (joinPartIndex == 0) {\n          // We have walked through all of the joinParts and have matches, so time to add to rowsToInclude\n\n          // We have ranges in each of the parts, we will do the cartesian product, so we will produce the product\n          // of the length of those ranges number of rows.  Let's compute it to see how many rows we will produce.\n          int numRowsExpected = joinPart.scanToRowIndex - joinPart.currRowIndex;\n          for (int i = 1; i < joinParts.size(); ++i) {\n            final JoinPart subPart = joinParts.get(i);\n            numRowsExpected *= subPart.scanToRowIndex - subPart.currRowIndex;\n          }\n\n          if (numRowsExpected == 1) {\n            for (int i = 0; i < joinParts.size(); ++i) {\n              rowsToInclude[i].add(joinParts.get(i).currRowIndex);\n            }\n          } else {\n            if (numRowsExpected > 1_000_000) {\n              // It would be helpful to serialize the actual value out with this error, but that risks leaking data\n              throw new IAE(\"Got a join, with a cartesian product that exceeds 1,000,000 rows, cannot handle it\");\n            }\n\n            // The rowIds that will be used in the result of the join will be a cartesian product, which means that\n            // the \"deepest\" row ids will be repeated in-order over and over, then the next layer will have each\n            // value repeated runSize times, forming a new run of length * runSize, and so on and so forth\n            int partIndex = joinParts.size() - 1;\n            int runSize = 1;\n            // We are guaranteed that there is a runSize greater than 1 because otherwise numRowsExpected would be 1\n            while (partIndex >= 0) {\n              final JoinPart part = joinParts.get(partIndex);\n              final int size = part.scanToRowIndex - part.currRowIndex;\n              if (size == 1) {\n                rowsToInclude[partIndex].fill(part.currRowIndex, numRowsExpected);\n              } else {\n                int[] vals = new int[size];\n                for (int i = 0; i < vals.length; ++i) {\n                  vals[i] = i + part.currRowIndex;\n                }","sourceCodeStart":339,"sourceCodeEnd":375,"githubUrl":"https://github.com/apache/druid/blob/9b90983fd291f26935af934383ce360473179e4d/processing/src/main/java/org/apache/druid/query/operator/join/SortedInnerJoinOperator.java#L339-L375","documentation":"During a sorted inner join, when join key values are equal across all inputs but the combined cross-product of matching rows would exceed 1,000,000 rows, Druid refuses to materialize the join. The library enforces this hard cap because a huge cartesian product would exhaust memory and stall the query. The message deliberately omits the offending values to avoid leaking data.","triggerScenarios":"joinRows (also reached via process/alternativeRow) encounters rows on all inputs whose join keys match, and the product of the number of matching rows per join part (numRowsExpected) exceeds 1,000,000.","commonSituations":"Joining on a low-cardinality key (e.g. boolean, status flag, or constant column) so one key value matches millions of rows; accidental cartesian join caused by a wrong or constant join condition; skewed data where one tenant/ID dominates.","solutions":["Fix the join condition so keys are high-cardinality and actually correlate the two inputs","Filter or pre-aggregate the inputs to reduce rows sharing a single key value","Reformulate as a GROUP BY or subquery to avoid the cross product","Increase limits by splitting the query into smaller partitions if the join is legitimately needed"],"exampleFix":"// before: join on constant/low-cardinality column\nJOIN dim ON (fact.partition = dim.partition)  // both have millions of rows per value\n// after: join on a selective key\nJOIN dim ON (fact.user_id = dim.user_id)","handlingStrategy":"validation","validationCode":"// before issuing the join, check key cardinality per value\nlong maxRowsPerKey = computeMaxRowsPerJoinKey(factRows, joinKey); // via GROUP BY key count()\nif (maxRowsPerKey * matchingDimRows > 1_000_000L) {\n  throw new IllegalArgumentException(\"join key too low-cardinality; refactor query\");\n}","typeGuard":"boolean isSafeJoin(DataSource left, DataSource right, JoinCondition c) {\n  return !c.getCondition().isAlwaysTrue() && estimateJoinCardinality(left, right, c) <= 1_000_000L;\n}","tryCatchPattern":"try {\n  runJoin(query);\n} catch (IllegalArgumentException e) {\n  if (e.getMessage().contains(\"cartesian product that exceeds\")) {\n    // fall back to pre-aggregated or filtered join\n    runJoin(refactorQuery(query));\n  } else throw e;\n}","preventionTips":["Join on high-cardinality keys, never on booleans/status/constant columns","Run a GROUP BY key COUNT(*) to detect skewed keys before joining","Never let a join condition degenerate to a tautology (e.g. 1=1)"],"tags":["join","cartesian-product","memory","query-processing"],"backgroundTag":"value-out-of-range","analyzedSha":"9b90983fd291f26935af934383ce360473179e4d","analyzedAt":"2026-09-07T13:32:30.957Z","contentChangedAt":"2026-09-07T13:32:30.957Z","schemaVersion":2},"datasetVersion":"2026-09-17T15:17:12.973Z"}