{"record":{"id":"1c70768231d1f232","repo":"apache/druid","slug":"configured-maxscattergatherbytes-d-is-more-than","errorCode":null,"errorMessage":"Configured maxScatterGatherBytes = %d is more than enforced limit of %d.","messagePattern":"Configured maxScatterGatherBytes = (.+?) is more than enforced limit of (.+?)\\.","errorType":"validation","errorClass":"BadQueryContextException","httpStatus":400,"severity":"error","filePath":"processing/src/main/java/org/apache/druid/query/QueryContext.java","lineNumber":586,"sourceCode":"    throw new BadQueryContextException(\n        StringUtils.format(\n            \"Per-segment timeout [%s] must be a non negative value, but was [%d]\",\n            QueryContexts.PER_SEGMENT_TIMEOUT_KEY,\n            timeout\n        )\n    );\n  }\n\n  public boolean usePerSegmentTimeout()\n  {\n    return getPerSegmentTimeout() != QueryContexts.NO_TIMEOUT;\n  }\n\n  public void verifyMaxScatterGatherBytes(long maxScatterGatherBytesLimit)\n  {\n    long curr = getLong(QueryContexts.MAX_SCATTER_GATHER_BYTES_KEY, 0);\n    if (curr > maxScatterGatherBytesLimit) {\n      throw new BadQueryContextException(\n          StringUtils.format(\n            \"Configured %s = %d is more than enforced limit of %d.\",\n            QueryContexts.MAX_SCATTER_GATHER_BYTES_KEY,\n            curr,\n            maxScatterGatherBytesLimit\n          )\n      );\n    }\n  }\n\n  public int getNumRetriesOnMissingSegments(int defaultValue)\n  {\n    return getInt(QueryContexts.NUM_RETRIES_ON_MISSING_SEGMENTS_KEY, defaultValue);\n  }\n\n  public boolean allowReturnPartialResults(boolean defaultValue)\n  {\n    return getBoolean(QueryContexts.RETURN_PARTIAL_RESULTS_KEY, defaultValue);","sourceCodeStart":568,"sourceCodeEnd":604,"githubUrl":"https://github.com/apache/druid/blob/9b90983fd291f26935af934383ce360473179e4d/processing/src/main/java/org/apache/druid/query/QueryContext.java#L568-L604","documentation":"Druid enforces an upper bound on `maxScatterGatherBytes` — the maximum number of bytes a query may buffer while gathering results from data nodes. QueryContext.verifyMaxScatterGatherBytes throws BadQueryContextException when the query's context value exceeds the enforced server limit. This protects brokers from unbounded memory use during fan-out query execution.","triggerScenarios":"Calling QueryContext.verifyMaxScatterGatherBytes(long maxScatterGatherBytesLimit) (invoked via withMaxScatterGatherBytes) when the context's MAX_SCATTER_GATHER_BYTES_KEY (\"maxScatterGatherBytes\") value exceeds the enforced limit, e.g. query asks for 10 GiB while the broker enforces 1 GiB.","commonSituations":"Clients inflating maxScatterGatherBytes to run very large group-bys/top-N queries on a broker whose druid.server.http.maxScatterGatherBytes or maxQueryTimeout-style cap was configured lower; operators tightening limits after capacity incidents; large-result BI tools with hard-coded generous limits.","solutions":["Reduce the query's `maxScatterGatherBytes` to a value at or below the enforced limit shown in the message.","If the workload legitimately needs more memory, raise the server-side enforced limit (broker's maxScatterGatherBytes config) and restart/roll out.","Restructure the query to return fewer bytes (filters, limits, approximate sketches like DataSketches theta/HLL) instead of raising limits.","Catch BadQueryContextException and surface the enforced limit to the user so they can shrink the query or contact the operator."],"exampleFix":"// before\ncontext.put(\"maxScatterGatherBytes\", 10L * 1024 * 1024 * 1024); // exceeds 1 GiB limit\n// after\nlong limit = 1024L * 1024 * 1024; // enforced by broker\ncontext.put(\"maxScatterGatherBytes\", Math.min(10L * 1024 * 1024 * 1024, limit));","handlingStrategy":"validation","validationCode":"Object msg = query.getContext().get(\"maxScatterGatherBytes\");\nlong enforced = /* broker config */ 1024L * 1024 * 1024;\nif (msg instanceof Number && ((Number) msg).longValue() > enforced) {\n    query.getContext().put(\"maxScatterGatherBytes\", enforced);\n}","typeGuard":"boolean withinScatterGatherLimit(Object v, long limit) {\n    return !(v instanceof Number) || ((Number) v).longValue() <= limit;\n}","tryCatchPattern":"try {\n    client.query(query);\n} catch (BadQueryContextException e) {\n    if (e.getMessage().contains(\"maxScatterGatherBytes\")) {\n        long limit = parseEnforcedLimit(e.getMessage());\n        query.getContext().put(\"maxScatterGatherBytes\", limit);\n        client.query(query);\n    } else {\n        throw e;\n    }\n}","preventionTips":["Only raise maxScatterGatherBytes with explicit operator approval; it maps to broker memory.","Reduce result size (filters, LIMIT, sketches) before raising byte limits.","Keep client-side caps in sync with broker configuration.","Alert on queries rejected for byte-limit breaches — they indicate oversized query patterns."],"tags":["query-context","memory-limit","scatter-gather","broker"],"backgroundTag":"value-out-of-range","analyzedSha":"9b90983fd291f26935af934383ce360473179e4d","analyzedAt":"2026-09-07T13:32:30.957Z","contentChangedAt":"2026-09-07T13:32:30.957Z","schemaVersion":2},"datasetVersion":"2026-09-17T15:17:12.973Z"}