apache/druid · error · IllegalArgumentException

Cannot pipeline with durable storage

Error message

Cannot pipeline with durable storage

What it means

Pipelined stages cannot use durable storage for intermediate results; the constructor throws IAE when both pipeline and durableStorage are true. Pipelining keeps stage output in memory between workers, which is incompatible with durably storing intermediate results.

Source

Thrown at multi-stage-query/src/main/java/org/apache/druid/msq/kernel/controller/ControllerQueryKernelConfig.java:78

  {
    if (maxRetainedPartitionSketchBytes <= 0) {
      throw new IAE("maxRetainedPartitionSketchBytes must be positive");
    }

    if (pipeline && maxConcurrentStages < 2) {
      throw new IAE("maxConcurrentStagesPerWorker must be >= 2 when pipelining");
    }

    if (maxConcurrentStages <= 0) {
      throw new IAE("maxConcurrentStagesPerWorker must be positive");
    }

    if (pipeline && faultTolerance) {
      throw new IAE("Cannot pipeline with fault tolerance");
    }

    if (pipeline && durableStorage) {
      throw new IAE("Cannot pipeline with durable storage");
    }

    if (faultTolerance && !durableStorage) {
      throw new IAE("Cannot have fault tolerance without durable storage");
    }

    this.maxRetainedPartitionSketchBytes = maxRetainedPartitionSketchBytes;
    this.maxConcurrentStages = maxConcurrentStages;
    this.pipeline = pipeline;
    this.durableStorage = durableStorage;
    this.faultTolerance = faultTolerance;
    this.destination = destination;
    this.controllerHost = controllerHost;
    this.workerIds = workerIds;
    this.workerContextMap = workerContextMap;
  }

  public static Builder builder()

View on GitHub (pinned to 9b90983fd2)

Solutions

1. Turn off durable storage for pipelined queries (remove durablyStoreTempResults / intermediate storage settings).
2. Or disable pipelining if durable intermediate storage is required.
3. Enforce exclusivity in code that builds the query context before submission.

Example fix

// before
context.put("pipelinedStagesEnabled", true);
context.put("durablyStoreTempResults", true);

// after
context.put("pipelinedStagesEnabled", true);
context.put("durablyStoreTempResults", false);

if (ctx.getBoolean("pipelinedStagesEnabled") && ctx.getBoolean("durablyStoreTempResults")) {
  throw new IllegalArgumentException("Pipelined queries cannot use durable storage for intermediates");
}

try {
  submitQuery(ctx);
} catch (IllegalArgumentException e) {
  if (e.getMessage().contains("Cannot pipeline with durable storage")) {
    ctx.put("durablyStoreTempResults", false);
    submitQuery(ctx);
  }
}

When it happens

Trigger: Constructing ControllerQueryKernelConfig with pipeline=true and durableStorage=true, e.g. query context combining pipelining with durablyStoreTempResults / druid.msq.intermediate.storage.enabled-type durable storage settings.

Common situations: Cluster-level durable storage configured for fault tolerance plus a per-query pipelining flag; users copying durable-storage tuning blocks into pipelined queries.

Understand the failure class

Background: "mutually exclusive" flag errors: what "can't supply both nx and xx", "--raw is not compatible with -i" and "cannot be used with" mean, and how to fix them — this error's family across 29 libraries.

Related errors

• Cannot pipeline with fault tolerance
• maxConcurrentStagesPerWorker must be >= 2 when pipelining
• Cannot have fault tolerance without durable storage
• NotEnoughMemoryFault
• Invalid value of %s.maxThreads[%d]

AI-assisted analysis of apache/druid@9b90983fd2 (2026-09-07). Data as JSON: /api/errors/32534677b0299c34. Report an issue: GitHub.