apache/druid · error · IAE

%s cannot be used for perfect rollup

Error message

%s cannot be used for perfect rollup

What it means

When the tuningConfig requests perfect rollup (forceGuaranteedRollup=true), the partitionsSpec must be of a type compatible with perfect rollup (isForceGuaranteedRollupCompatibleType(), i.e., single-dim/range or hash with fixed numShards — not dynamic sizing). If incompatible, the IndexTask constructor throws this IAE naming the partitionsSpec class. Perfect rollup requires knowing shard counts up front, which dynamic partitions cannot provide.

Source

Thrown at indexing-service/src/main/java/org/apache/druid/indexing/common/task/IndexTask.java:1257

        if (forceGuaranteedRollup) {
          if (maxRowsPerSegment != null
              || numShards != null
              || (partitionDimensions != null && !partitionDimensions.isEmpty())) {
            return new HashedPartitionsSpec(maxRowsPerSegment, numShards, partitionDimensions);
          } else {
            return null;
          }
        } else {
          if (maxRowsPerSegment != null || maxTotalRows != null) {
            return new DynamicPartitionsSpec(maxRowsPerSegment, maxTotalRows);
          } else {
            return null;
          }
        }
      } else {
        if (forceGuaranteedRollup) {
          if (!partitionsSpec.isForceGuaranteedRollupCompatibleType()) {
            throw new IAE(partitionsSpec.getClass().getSimpleName() + " cannot be used for perfect rollup");
          }
        } else {
          if (!(partitionsSpec instanceof DynamicPartitionsSpec)) {
            throw new IAE("DynamicPartitionsSpec must be used for best-effort rollup");
          }
        }
        return partitionsSpec;
      }
    }

    @JsonCreator
    public IndexTuningConfig(
        @JsonProperty("targetPartitionSize") @Deprecated @Nullable Integer targetPartitionSize,
        @JsonProperty("maxRowsPerSegment") @Deprecated @Nullable Integer maxRowsPerSegment,
        @JsonProperty("appendableIndexSpec") @Nullable AppendableIndexSpec appendableIndexSpec,
        @JsonProperty("maxRowsInMemory") @Nullable Integer maxRowsInMemory,
        @JsonProperty("maxBytesInMemory") @Nullable Long maxBytesInMemory,
        @JsonProperty("skipBytesInMemoryOverheadCheck") @Nullable Boolean skipBytesInMemoryOverheadCheck,

View on GitHub (pinned to 9b90983fd2)

Solutions

  1. Change partitionsSpec to type "hash" with a fixed "numShards" (or single_dim/range) when forceGuaranteedRollup is true
  2. Alternatively drop forceGuaranteedRollup to use best-effort rollup with dynamic partitions
  3. Ensure partitionKeys/hash partitioning is specified so shard counts are deterministic
  4. Run a separate pass: ingest best-effort, then compact with fixed partitions for perfect rollup

Example fix

// before
"tuningConfig": {
  "type": "index_parallel",
  "forceGuaranteedRollup": true,
  "partitionsSpec": { "type": "linear", "targetRowsPerSegment": 5000000 }
}
// after
"tuningConfig": {
  "type": "index_parallel",
  "forceGuaranteedRollup": true,
  "partitionsSpec": { "type": "hash", "numShards": 4 }
}
Defensive patterns

Strategy: validation

Validate before calling

if (Boolean.TRUE.equals(tuningConfig.getForceGuaranteedRollup())
    && !tuningConfig.getPartitionsSpec().isForceGuaranteedRollupCompatibleType()) {
  throw new IllegalArgumentException("Perfect rollup needs hash/single_dim partitionsSpec with fixed shards");
}

Type guard

boolean perfectRollupCompatible(PartitionsSpec p) {
  return p != null && p.isForceGuaranteedRollupCompatibleType();
}

Try / catch

try {
  IndexTask task = buildIndexTask(schema, tuningConfig);
} catch (IllegalArgumentException e) {
  if (e.getMessage().endsWith("cannot be used for perfect rollup")) {
    tuningConfig = tuningConfig.withPartitionsSpec(hashSpecWithFixedShards());
    task = buildIndexTask(schema, tuningConfig);
  } else throw e;
}

Prevention

When it happens

Trigger: Setting "forceGuaranteedRollup": true (or building tuningConfig with perfect-rollup semantics) while using DynamicPartitionsSpec (type "linear", possibly with maxTotalRows), or any partitionsSpec whose isForceGuaranteedRollupCompatibleType() returns false.

Common situations: Users wanting deduplication/perfect rollup leaving partitionsSpec as default linear; copy-pasting a normal batch tuningConfig and adding forceGuaranteedRollup without changing partitionsSpec; older specs migrated to new tuningConfig format.

Related errors

AI-assisted analysis of apache/druid@9b90983fd2 (2026-09-07). Data as JSON: /api/errors/fc0a2d71bcaac3c2. Report an issue: GitHub.