apache/druid · error · IllegalStateException (ISE)

Cannot apply limit[%d] with offset[%d] due to overflow

Error message

Cannot apply limit[%d] with offset[%d] due to overflow

What it means

When merging scan results across segments/distributed sub-queries, the tool chest folds the query's offset into the effective limit (newLimit = limit + offset). If that addition overflows Long (limit > Long.MAX_VALUE - offset), it throws ISE because the merged limit cannot be represented.

Source

Thrown at processing/src/main/java/org/apache/druid/query/scan/ScanQueryQueryToolChest.java:83

  {
    this.queryMetricsFactory = queryMetricsFactory;
  }

  @Override
  public QueryRunner<ScanResultValue> mergeResults(final QueryRunner<ScanResultValue> runner)
  {
    return (queryPlus, responseContext) -> {
      final ScanQuery originalQuery = ((ScanQuery) (queryPlus.getQuery()));
      ScanQuery.verifyOrderByForNativeExecution(originalQuery);

      // Remove "offset" and add it to the "limit" (we won't push the offset down, just apply it here, at the
      // merge at the top of the stack).
      final long newLimit;
      if (!originalQuery.isLimited()) {
        // Unlimited stays unlimited.
        newLimit = Long.MAX_VALUE;
      } else if (originalQuery.getScanRowsLimit() > Long.MAX_VALUE - originalQuery.getScanRowsOffset()) {
        throw new ISE(
            "Cannot apply limit[%d] with offset[%d] due to overflow",
            originalQuery.getScanRowsLimit(),
            originalQuery.getScanRowsOffset()
        );
      } else {
        newLimit = originalQuery.getScanRowsLimit() + originalQuery.getScanRowsOffset();
      }

      final ScanQuery queryToRun = originalQuery.withOffset(0)
                                                .withLimit(newLimit);

      final Sequence<ScanResultValue> results;

      if (!queryToRun.isLimited()) {
        results = runner.run(queryPlus.withQuery(queryToRun), responseContext);
      } else {
        results = new BaseSequence<>(
            new BaseSequence.IteratorMaker<ScanResultValue, ScanQueryLimitRowIterator>()

View on GitHub (pinned to 9b90983fd2)

Solutions

1. Reduce the scanRows limit to a sane value so limit+offset fits in a long
2. Remove the offset, or drop the limit so the query is unlimited
3. Clamp limit client-side, e.g. Math.min(limit, Long.MAX_VALUE - offset)

Example fix

// before
{"queryType":"scan","limit":9223372036854775807,"offset":100,...}
// after
{"queryType":"scan","limit":1000000,"offset":100,...}

if (query.isLimited() && query.getScanRowsLimit() > Long.MAX_VALUE - query.getScanRowsOffset()) {
  throw new IllegalArgumentException("limit+offset overflows; lower the limit or offset");
}

boolean limitOffsetSafe(ScanQuery q) {
  return !q.isLimited() || q.getScanRowsLimit() <= Long.MAX_VALUE - q.getScanRowsOffset();
}

try {
  chest.mergeResults(query, seq, responseContext);
} catch (IllegalStateException e) {
  if (e.getMessage().contains("due to overflow")) { /* resubmit with clamped limit */ }
  else throw e;
}

When it happens

Trigger: Calling ScanQueryQueryToolChest.mergeResults on a query with isLimited() true and getScanRowsLimit() so large that limit + getScanRowsOffset() exceeds Long.MAX_VALUE.

Common situations: Clients sending an astronomically large scanRowsLimit (e.g. Long.MAX_VALUE) together with a nonzero offset; programmatic query builders copying sentinel max-limit values and then adding pagination offsets.

Related errors

• 'skip' must be greater than zero
• Cannot apply limit[%d] with offset[%d] due to overflow
• Column [%s] from 'orderBy' must also appear in 'columns'.
• The __time column must be selected if the results are time-o
• Cannot execute query with orderBy %s

AI-assisted analysis of apache/druid@9b90983fd2 (2026-09-07). Data as JSON: /api/errors/1c8f5614b5676217. Report an issue: GitHub.