{"record":{"id":"a6ef7c8b05fcccb1","repo":"apache/druid","slug":"query-timeout-cancelling-pending-results-for-quer-a6ef7c","errorCode":null,"errorMessage":"Query timeout, cancelling pending results for query [%s]. Per-segment timeout exceeded.","messagePattern":"Query timeout, cancelling pending results for query \\[(.+?)\\]\\. Per-segment timeout exceeded\\.","errorType":"exception","errorClass":"QueryTimeoutException","httpStatus":null,"severity":"error","filePath":"processing/src/main/java/org/apache/druid/query/groupby/epinephelinae/GroupByMergingQueryRunner.java","lineNumber":407,"sourceCode":"      }\n    }\n    catch (InterruptedException | CancellationException e) {\n      log.noStackTrace().warn(e, \"Query interrupted, cancelling pending results for query [%s]\", query.getId());\n      GuavaUtils.cancelAll(true, future, futures);\n      throw new QueryInterruptedException(e);\n    }\n    catch (QueryTimeoutException | TimeoutException e) {\n      log.noStackTrace().warn(e, \"Query timeout, cancelling pending results for query [%s]\", query.getId());\n      GuavaUtils.cancelAll(true, future, futures);\n      throw new QueryTimeoutException(StringUtils.nonStrictFormat(\"Query [%s] timed out\", query.getId()));\n    }\n    catch (ExecutionException e) {\n      log.noStackTrace().warn(e, \"Query error, cancelling pending results for query [%s]\", query.getId());\n      GuavaUtils.cancelAll(true, future, futures);\n      Throwable cause = e.getCause();\n      // Nested per-segment future timeout\n      if (cause instanceof TimeoutException) {\n        throw new QueryTimeoutException(StringUtils.nonStrictFormat(\"Query timeout, cancelling pending results for query [%s]. Per-segment timeout exceeded.\", query.getId()));\n      }\n      throw new RuntimeException(e);\n    }\n  }\n}\n","sourceCodeStart":389,"sourceCodeEnd":413,"githubUrl":"https://github.com/apache/druid/blob/9b90983fd291f26935af934383ce360473179e4d/processing/src/main/java/org/apache/druid/query/groupby/epinephelinae/GroupByMergingQueryRunner.java#L389-L413","documentation":"When a per-segment future fails with an ExecutionException whose cause is a TimeoutException, waitForFutureCompletion rethrows it as QueryTimeoutException with the 'Per-segment timeout exceeded' message, distinguishing it from a whole-query timeout. The pending futures are cancelled first.","triggerScenarios":"Individual segment processing exceeded the per-segment timeout (query context perSegmentTimeout / segment-level TimeoutException from PrioritizedQueryRunnerCallable), wrapped into ExecutionException, detected in waitForFutureCompletion's catch block.","commonSituations":"One very large or cold-cached segment taking longer than the per-segment timeout; data skew where a single segment dominates runtime; per-segment timeout configured below realistic segment scan times.","solutions":["Increase the per-segment timeout in the query context (e.g. timeout-related context keys) or the overall timeout","Investigate the slow segment: repurpose/compact it (smaller segments via compaction), warm its cache","Add pre-filtering or reduce aggregation cardinality for the affected interval","Check historical server health (GC pauses, disk IO) for that segment's host","Re-run the query; if consistently hitting one segment, re-balance or compact"],"exampleFix":"// before\ncontext: {\"timeout\":\"300000\"} // per-segment work exceeds limit\n// after\ncontext: {\"timeout\":\"600000\", \"maxScatterGatherBytes\": ...} // raise timeout + compact slow segments","handlingStrategy":"retry","validationCode":"// before submitting: ensure per-segment timeout exceeds slowest segment scan estimate\nif (slowestSegmentScanMs > perSegmentTimeoutMs) { raiseTimeoutOrCompactSegments(); }","typeGuard":null,"tryCatchPattern":"try { runQuery(query); } catch (QueryTimeoutException e) { if (e.getMessage().contains(\"Per-segment timeout exceeded\")) { compactSlowSegments(); retry(query); } else { throw e; } }","preventionTips":["Enable compaction to avoid oversized/skewed segments","Warm caches for hot segments","Set per-segment timeouts above worst-case single-segment scan time"],"tags":["druid","groupby","timeout","per-segment","distributed"],"backgroundTag":"request-timeout","analyzedSha":"9b90983fd291f26935af934383ce360473179e4d","analyzedAt":"2026-09-07T13:32:30.957Z","contentChangedAt":"2026-09-07T13:32:30.957Z","schemaVersion":2},"datasetVersion":"2026-09-14T05:17:10.506Z"}