apache/druid · error · IAE

DynamicPartitionsSpec must be used for best-effort rollup

Error message

DynamicPartitionsSpec must be used for best-effort rollup

What it means

IndexTask validates that the configured partitionsSpec matches the requested rollup mode. Perfect (guaranteed) rollup requires a hash/single-dimension partitionsSpec, while best-effort rollup (forceGuaranteedRollup=false) only supports DynamicPartitionsSpec. Throwing IAE fails fast at task setup instead of producing silently skewed segments.

Source

Thrown at indexing-service/src/main/java/org/apache/druid/indexing/common/task/IndexTask.java:1261

            return new HashedPartitionsSpec(maxRowsPerSegment, numShards, partitionDimensions);
          } else {
            return null;
          }
        } else {
          if (maxRowsPerSegment != null || maxTotalRows != null) {
            return new DynamicPartitionsSpec(maxRowsPerSegment, maxTotalRows);
          } else {
            return null;
          }
        }
      } else {
        if (forceGuaranteedRollup) {
          if (!partitionsSpec.isForceGuaranteedRollupCompatibleType()) {
            throw new IAE(partitionsSpec.getClass().getSimpleName() + " cannot be used for perfect rollup");
          }
        } else {
          if (!(partitionsSpec instanceof DynamicPartitionsSpec)) {
            throw new IAE("DynamicPartitionsSpec must be used for best-effort rollup");
          }
        }
        return partitionsSpec;
      }
    }

    @JsonCreator
    public IndexTuningConfig(
        @JsonProperty("targetPartitionSize") @Deprecated @Nullable Integer targetPartitionSize,
        @JsonProperty("maxRowsPerSegment") @Deprecated @Nullable Integer maxRowsPerSegment,
        @JsonProperty("appendableIndexSpec") @Nullable AppendableIndexSpec appendableIndexSpec,
        @JsonProperty("maxRowsInMemory") @Nullable Integer maxRowsInMemory,
        @JsonProperty("maxBytesInMemory") @Nullable Long maxBytesInMemory,
        @JsonProperty("skipBytesInMemoryOverheadCheck") @Nullable Boolean skipBytesInMemoryOverheadCheck,
        @JsonProperty("maxTotalRows") @Deprecated @Nullable Long maxTotalRows,
        @JsonProperty("rowFlushBoundary") @Deprecated @Nullable Integer rowFlushBoundary_forBackCompatibility,
        @JsonProperty("numShards") @Deprecated @Nullable Integer numShards,
        @JsonProperty("partitionDimensions") @Deprecated @Nullable List<String> partitionDimensions,

View on GitHub (pinned to 9b90983fd2)

Solutions

1. Set partitionsSpec to a dynamic spec: {"type":"dynamic","maxTotalRows":...} when forceGuaranteedRollup is false
2. Alternatively set forceGuaranteedRollup=true if perfect rollup with the existing hash/single-dim partitionsSpec is intended
3. Regenerate the task spec from the ingestion wizard for the target rollup mode

Example fix

// before
"forceGuaranteedRollup": false,
"partitionsSpec": {"type": "hashed", "numShards": 4}
// after
"forceGuaranteedRollup": false,
"partitionsSpec": {"type": "dynamic", "maxTotalRows": 5000000}

if (!taskSpec.getTuningConfig().getPartitionsSpec().isForceGuaranteedRollupCompatibleType() && !taskSpec.isForceGuaranteedRollup()) { throw new IllegalArgumentException("Use dynamic partitionsSpec for best-effort rollup"); }

try { taskClient.submit(spec); } catch (IllegalArgumentException e) { if (e.getMessage().contains("best-effort rollup")) { spec = fixPartitionsSpec(spec); } else { throw e; } }

When it happens

Trigger: Submitting an IndexTask with forceGuaranteedRollup=false (the default) but a partitionsSpec of HashedPartitionsSpec or SingleDimensionPartitionsSpec instead of DynamicPartitionsSpec.

Common situations: Copying a perfect-rollup task spec and flipping forceGuaranteedRollup to false without switching partitionsSpec; upgrading from older Druid where hashed partitions were allowed more freely; ingestion-platform templates that always emit a static partitionsSpec.

Understand the failure class

Background: "is not a compatible type" / "cannot merge" errors: when a value's type doesn't match what the library requires — this error's family across 65 libraries.

Related errors

• Invalid bucketByCount [%d]
• Cannot mix sortable and unsortable key columns
• Must provide at least one range
• Aggregation [%s] does not support column [%s] of type [%s].
• Cannot accept both 'splitPoints' and 'numBins'

AI-assisted analysis of apache/druid@9b90983fd2 (2026-09-07). Data as JSON: /api/errors/c097b0551296f463. Report an issue: GitHub.