{"record":{"id":"fc0a2d71bcaac3c2","repo":"apache/druid","slug":"s-cannot-be-used-for-perfect-rollup","errorCode":null,"errorMessage":"%s cannot be used for perfect rollup","messagePattern":"(.+?) cannot be used for perfect rollup","errorType":"validation","errorClass":"IAE","httpStatus":null,"severity":"error","filePath":"indexing-service/src/main/java/org/apache/druid/indexing/common/task/IndexTask.java","lineNumber":1257,"sourceCode":"        if (forceGuaranteedRollup) {\n          if (maxRowsPerSegment != null\n              || numShards != null\n              || (partitionDimensions != null && !partitionDimensions.isEmpty())) {\n            return new HashedPartitionsSpec(maxRowsPerSegment, numShards, partitionDimensions);\n          } else {\n            return null;\n          }\n        } else {\n          if (maxRowsPerSegment != null || maxTotalRows != null) {\n            return new DynamicPartitionsSpec(maxRowsPerSegment, maxTotalRows);\n          } else {\n            return null;\n          }\n        }\n      } else {\n        if (forceGuaranteedRollup) {\n          if (!partitionsSpec.isForceGuaranteedRollupCompatibleType()) {\n            throw new IAE(partitionsSpec.getClass().getSimpleName() + \" cannot be used for perfect rollup\");\n          }\n        } else {\n          if (!(partitionsSpec instanceof DynamicPartitionsSpec)) {\n            throw new IAE(\"DynamicPartitionsSpec must be used for best-effort rollup\");\n          }\n        }\n        return partitionsSpec;\n      }\n    }\n\n    @JsonCreator\n    public IndexTuningConfig(\n        @JsonProperty(\"targetPartitionSize\") @Deprecated @Nullable Integer targetPartitionSize,\n        @JsonProperty(\"maxRowsPerSegment\") @Deprecated @Nullable Integer maxRowsPerSegment,\n        @JsonProperty(\"appendableIndexSpec\") @Nullable AppendableIndexSpec appendableIndexSpec,\n        @JsonProperty(\"maxRowsInMemory\") @Nullable Integer maxRowsInMemory,\n        @JsonProperty(\"maxBytesInMemory\") @Nullable Long maxBytesInMemory,\n        @JsonProperty(\"skipBytesInMemoryOverheadCheck\") @Nullable Boolean skipBytesInMemoryOverheadCheck,","sourceCodeStart":1239,"sourceCodeEnd":1275,"githubUrl":"https://github.com/apache/druid/blob/9b90983fd291f26935af934383ce360473179e4d/indexing-service/src/main/java/org/apache/druid/indexing/common/task/IndexTask.java#L1239-L1275","documentation":"When the tuningConfig requests perfect rollup (forceGuaranteedRollup=true), the partitionsSpec must be of a type compatible with perfect rollup (isForceGuaranteedRollupCompatibleType(), i.e., single-dim/range or hash with fixed numShards — not dynamic sizing). If incompatible, the IndexTask constructor throws this IAE naming the partitionsSpec class. Perfect rollup requires knowing shard counts up front, which dynamic partitions cannot provide.","triggerScenarios":"Setting \"forceGuaranteedRollup\": true (or building tuningConfig with perfect-rollup semantics) while using DynamicPartitionsSpec (type \"linear\", possibly with maxTotalRows), or any partitionsSpec whose isForceGuaranteedRollupCompatibleType() returns false.","commonSituations":"Users wanting deduplication/perfect rollup leaving partitionsSpec as default linear; copy-pasting a normal batch tuningConfig and adding forceGuaranteedRollup without changing partitionsSpec; older specs migrated to new tuningConfig format.","solutions":["Change partitionsSpec to type \"hash\" with a fixed \"numShards\" (or single_dim/range) when forceGuaranteedRollup is true","Alternatively drop forceGuaranteedRollup to use best-effort rollup with dynamic partitions","Ensure partitionKeys/hash partitioning is specified so shard counts are deterministic","Run a separate pass: ingest best-effort, then compact with fixed partitions for perfect rollup"],"exampleFix":"// before\n\"tuningConfig\": {\n  \"type\": \"index_parallel\",\n  \"forceGuaranteedRollup\": true,\n  \"partitionsSpec\": { \"type\": \"linear\", \"targetRowsPerSegment\": 5000000 }\n}\n// after\n\"tuningConfig\": {\n  \"type\": \"index_parallel\",\n  \"forceGuaranteedRollup\": true,\n  \"partitionsSpec\": { \"type\": \"hash\", \"numShards\": 4 }\n}","handlingStrategy":"validation","validationCode":"if (Boolean.TRUE.equals(tuningConfig.getForceGuaranteedRollup())\n    && !tuningConfig.getPartitionsSpec().isForceGuaranteedRollupCompatibleType()) {\n  throw new IllegalArgumentException(\"Perfect rollup needs hash/single_dim partitionsSpec with fixed shards\");\n}","typeGuard":"boolean perfectRollupCompatible(PartitionsSpec p) {\n  return p != null && p.isForceGuaranteedRollupCompatibleType();\n}","tryCatchPattern":"try {\n  IndexTask task = buildIndexTask(schema, tuningConfig);\n} catch (IllegalArgumentException e) {\n  if (e.getMessage().endsWith(\"cannot be used for perfect rollup\")) {\n    tuningConfig = tuningConfig.withPartitionsSpec(hashSpecWithFixedShards());\n    task = buildIndexTask(schema, tuningConfig);\n  } else throw e;\n}","preventionTips":["Pair forceGuaranteedRollup=true only with fixed-shard hash or single_dim partitionsSpec","Never use type linear (DynamicPartitionsSpec) with perfect rollup","Lint tuningConfig pairs (rollup flag vs partitionsSpec) before submission","Document that perfect rollup requires upfront shard counts"],"tags":["druid","ingestion","rollup","partitionspec"],"backgroundTag":"conflicting-config-options","analyzedSha":"9b90983fd291f26935af934383ce360473179e4d","analyzedAt":"2026-09-07T13:32:30.957Z","contentChangedAt":"2026-09-07T13:32:30.957Z","schemaVersion":2},"datasetVersion":"2026-09-14T05:17:10.506Z"}