{"record":{"id":"c097b0551296f463","repo":"apache/druid","slug":"dynamicpartitionsspec-must-be-used-for-best-effort","errorCode":null,"errorMessage":"DynamicPartitionsSpec must be used for best-effort rollup","messagePattern":"DynamicPartitionsSpec must be used for best-effort rollup","errorType":"validation","errorClass":"IAE","httpStatus":null,"severity":"error","filePath":"indexing-service/src/main/java/org/apache/druid/indexing/common/task/IndexTask.java","lineNumber":1261,"sourceCode":"            return new HashedPartitionsSpec(maxRowsPerSegment, numShards, partitionDimensions);\n          } else {\n            return null;\n          }\n        } else {\n          if (maxRowsPerSegment != null || maxTotalRows != null) {\n            return new DynamicPartitionsSpec(maxRowsPerSegment, maxTotalRows);\n          } else {\n            return null;\n          }\n        }\n      } else {\n        if (forceGuaranteedRollup) {\n          if (!partitionsSpec.isForceGuaranteedRollupCompatibleType()) {\n            throw new IAE(partitionsSpec.getClass().getSimpleName() + \" cannot be used for perfect rollup\");\n          }\n        } else {\n          if (!(partitionsSpec instanceof DynamicPartitionsSpec)) {\n            throw new IAE(\"DynamicPartitionsSpec must be used for best-effort rollup\");\n          }\n        }\n        return partitionsSpec;\n      }\n    }\n\n    @JsonCreator\n    public IndexTuningConfig(\n        @JsonProperty(\"targetPartitionSize\") @Deprecated @Nullable Integer targetPartitionSize,\n        @JsonProperty(\"maxRowsPerSegment\") @Deprecated @Nullable Integer maxRowsPerSegment,\n        @JsonProperty(\"appendableIndexSpec\") @Nullable AppendableIndexSpec appendableIndexSpec,\n        @JsonProperty(\"maxRowsInMemory\") @Nullable Integer maxRowsInMemory,\n        @JsonProperty(\"maxBytesInMemory\") @Nullable Long maxBytesInMemory,\n        @JsonProperty(\"skipBytesInMemoryOverheadCheck\") @Nullable Boolean skipBytesInMemoryOverheadCheck,\n        @JsonProperty(\"maxTotalRows\") @Deprecated @Nullable Long maxTotalRows,\n        @JsonProperty(\"rowFlushBoundary\") @Deprecated @Nullable Integer rowFlushBoundary_forBackCompatibility,\n        @JsonProperty(\"numShards\") @Deprecated @Nullable Integer numShards,\n        @JsonProperty(\"partitionDimensions\") @Deprecated @Nullable List<String> partitionDimensions,","sourceCodeStart":1243,"sourceCodeEnd":1279,"githubUrl":"https://github.com/apache/druid/blob/9b90983fd291f26935af934383ce360473179e4d/indexing-service/src/main/java/org/apache/druid/indexing/common/task/IndexTask.java#L1243-L1279","documentation":"IndexTask validates that the configured partitionsSpec matches the requested rollup mode. Perfect (guaranteed) rollup requires a hash/single-dimension partitionsSpec, while best-effort rollup (forceGuaranteedRollup=false) only supports DynamicPartitionsSpec. Throwing IAE fails fast at task setup instead of producing silently skewed segments.","triggerScenarios":"Submitting an IndexTask with forceGuaranteedRollup=false (the default) but a partitionsSpec of HashedPartitionsSpec or SingleDimensionPartitionsSpec instead of DynamicPartitionsSpec.","commonSituations":"Copying a perfect-rollup task spec and flipping forceGuaranteedRollup to false without switching partitionsSpec; upgrading from older Druid where hashed partitions were allowed more freely; ingestion-platform templates that always emit a static partitionsSpec.","solutions":["Set partitionsSpec to a dynamic spec: {\"type\":\"dynamic\",\"maxTotalRows\":...} when forceGuaranteedRollup is false","Alternatively set forceGuaranteedRollup=true if perfect rollup with the existing hash/single-dim partitionsSpec is intended","Regenerate the task spec from the ingestion wizard for the target rollup mode"],"exampleFix":"// before\n\"forceGuaranteedRollup\": false,\n\"partitionsSpec\": {\"type\": \"hashed\", \"numShards\": 4}\n// after\n\"forceGuaranteedRollup\": false,\n\"partitionsSpec\": {\"type\": \"dynamic\", \"maxTotalRows\": 5000000}","handlingStrategy":"validation","validationCode":"if (!taskSpec.getTuningConfig().getPartitionsSpec().isForceGuaranteedRollupCompatibleType() && !taskSpec.isForceGuaranteedRollup()) { throw new IllegalArgumentException(\"Use dynamic partitionsSpec for best-effort rollup\"); }","typeGuard":null,"tryCatchPattern":"try { taskClient.submit(spec); } catch (IllegalArgumentException e) { if (e.getMessage().contains(\"best-effort rollup\")) { spec = fixPartitionsSpec(spec); } else { throw e; } }","preventionTips":["Match partitionsSpec.type to forceGuaranteedRollup whenever editing specs by hand","Use Druid's ingestion wizard to generate specs instead of copying old templates","Add a config lint in CI that validates rollup/partitionsSpec pairing"],"tags":["druid","indexing","configuration","validation"],"backgroundTag":"incompatible-source-type","analyzedSha":"9b90983fd291f26935af934383ce360473179e4d","analyzedAt":"2026-09-07T13:32:30.957Z","contentChangedAt":"2026-09-07T13:32:30.957Z","schemaVersion":2},"datasetVersion":"2026-09-14T05:17:10.506Z"}