apache/druid · error · ResourceLimitExceededException

Query[ ] url[ ] total bytes gathered[%,d] exceeds…

Error message

Query[%s] url[%s] total bytes gathered[%,d] exceeds maxScatterGatherBytes[%,d]

What it means

Thrown by checkTotalBytesLimit() in DirectDruidClient when the cumulative bytes gathered across all data nodes for a query exceed the maxScatterGatherBytes limit (from query context, bounded by druid.server.http.maxScatterGatherBytes). It is a ResourceLimitExceededException protecting the broker from unbounded memory use in scatter-gather.

Solutions

Raise maxScatterGatherBytes in the query context (bounded by the broker's druid.server.http.maxScatterGatherBytes) or increase the broker config.
Narrow the query: reduce intervals, add filters, project only needed columns, or page through results.
Export results incrementally rather than materializing everything in one query.
If the limit was hit unexpectedly, check for runaway queries (e.g. missing time filter).

Example fix

// before
// context: {} (uses low default maxScatterGatherBytes)
// after
// context: {"maxScatterGatherBytes": 1073741824}

Defensive patterns

Strategy: try-catch

Validate before calling

// estimate response size before issuing
long estBytes = estimateQueryResponseBytes(query); // columns * rows * avg width
long limit = query.context().get("maxScatterGatherBytes");
if (estBytes > limit) { narrowQuery(query); }

Try / catch

try {
  return client.run(query, ctx).toList();
} catch (ResourceLimitExceededException e) {
  // split query into smaller intervals or raise maxScatterGatherBytes
  throw e;
}

Prevention

Always include time filters; avoid unbounded scans.
Project only the columns needed.
Set maxScatterGatherBytes consciously per workload.
Break very large queries into interval chunks.

When it happens

Trigger: Running a query whose total response bytes from all historicals exceed maxScatterGatherBytes: unbounded scans/selects, huge group-by cardinalities, or select-all column queries over large intervals.

Common situations: Exploratory 'SELECT *' style queries over large time ranges; per-query limit left at a low default (1GiB) while users run large exports; limit set in cluster config too small for reporting workloads.

Understand the failure class

Background: payload too large / request exceeds maximum size: why libraries cap bytes and how to fix oversize payloads — this error's family across 50 libraries.

Related errors

AI-assisted analysis of apache/druid@9b90983fd2 (2026-09-07). Data as JSON: /api/errors/8b19d07d008bdebb. Report an issue: GitHub.

Appendix: source

Thrown at server/src/main/java/org/apache/druid/client/DirectDruidClient.java:500

            throw new QueryTimeoutException(msg);
          } else {
            return timeLeft;
          }
        }

        private void checkTotalBytesLimit(long bytes)
        {
          final long currentTotalBytesGathered = totalBytesGathered.addAndGet(bytes);
          if (currentTotalBytesGathered > maxScatterGatherBytes) {
            String msg = StringUtils.format(
                "Query[%s] url[%s] total bytes gathered[%,d] exceeds maxScatterGatherBytes[%,d]",
                query.getId(),
                url,
                currentTotalBytesGathered,
                maxScatterGatherBytes
            );
            setupResponseReadFailure(msg, null);
            throw new ResourceLimitExceededException(msg);
          }
        }
      };

      long timeLeft = timeoutAt - System.currentTimeMillis();

      if (timeLeft <= 0) {
        throw new QueryTimeoutException(StringUtils.nonStrictFormat("Query[%s] url[%s] timed out.", query.getId(), url));
      }

      // increment is moved up so that if future initialization is queued by some other process,
      // we can increment the count earlier so that we can route the request to a different server
      openConnections.getAndIncrement();
      try {
        future = httpClient.go(
            new Request(
                HttpMethod.POST,
                new URL(url)

View on GitHub (pinned to 9b90983fd2)