{"record":{"id":"1ba10af58163ec17","repo":"apache/druid","slug":"backfill-tasks-require-useconcurrentlocks-to-be","errorCode":null,"errorMessage":"Backfill tasks require 'useConcurrentLocks' to be set to true in the supervisor context to allow concurrent writes with the main supervisor tasks","messagePattern":"Backfill tasks require 'useConcurrentLocks' to be set to true in the supervisor context to allow concurrent writes with the main supervisor tasks","errorType":"validation","errorClass":"IllegalArgumentException","httpStatus":null,"severity":"error","filePath":"indexing-service/src/main/java/org/apache/druid/indexing/overlord/supervisor/SupervisorManager.java","lineNumber":516,"sourceCode":"\n    return ImmutableMap.of(\n        \"id\", id,\n        \"backfillSupervisorId\", backfillSupervisorId\n    );\n  }\n\n  private void validateResetAndBackfill(\n      String id,\n      SeekableStreamSupervisor streamSupervisor,\n      SeekableStreamSupervisorSpec streamSpec\n  )\n  {\n    if (streamSupervisor.getIoConfig().isUseEarliestSequenceNumber()) {\n      throw new IAE(\"Reset with skipped offsets is not supported when useEarliestOffset is true.\");\n    }\n\n    if (!specHasConcurrentLocks(streamSpec)) {\n      throw new IAE(\n          \"Backfill tasks require 'useConcurrentLocks' to be set to true in the supervisor context to allow concurrent writes with the main supervisor tasks\"\n      );\n    }\n\n    if (streamSupervisor.getState() != SupervisorStateManager.BasicState.RUNNING) {\n      throw new IAE(\"Supervisor[%s] must be in a RUNNING state to perform a reset and backfill\", id);\n    }\n  }\n\n  public boolean checkPointDataSourceMetadata(\n      String supervisorId,\n      int taskGroupId,\n      DataSourceMetadata previousDataSourceMetadata\n  )\n  {\n    try {\n      Preconditions.checkState(started, \"SupervisorManager not started\");\n      Preconditions.checkNotNull(supervisorId, \"supervisorId cannot be null\");","sourceCodeStart":498,"sourceCodeEnd":534,"githubUrl":"https://github.com/apache/druid/blob/9b90983fd291f26935af934383ce360473179e4d/indexing-service/src/main/java/org/apache/druid/indexing/overlord/supervisor/SupervisorManager.java#L498-L534","documentation":"SupervisorManager requires the supervisor context to enable concurrent locks before starting backfill tasks alongside the main supervisor tasks. Without useConcurrentLocks=true, backfill tasks would contend with main tasks for segment locks and the manager rejects the request with this message. The validation is part of validateResetAndBackfill and fires before any reset is performed.","triggerScenarios":"resetToLatestAndBackfill (POST /druid/indexer/v1/supervisor/<id>/reset with backfill) where specHasConcurrentLocks(streamSpec) is false — i.e. the supervisor context lacks useConcurrentLocks=true.","commonSituations":"Pre-existing supervisors created before concurrent locks were introduced, whose context never set the flag; spec copied from an old template; operator enabling backfill without reviewing the locking requirements.","solutions":["Add \"useConcurrentLocks\": true to the supervisor's context (suspend, POST updated spec, resume), then retry","If concurrent locks cannot be enabled, stop the supervisor and run backfill as a standalone batch ingestion task instead","Verify with GET /druid/indexer/v1/supervisor/<id> that the context field shows the updated flag"],"exampleFix":"// before\n{\"context\": {\"maxActiveTaskGroups\": 3}}\n// after\n{\"context\": {\"maxActiveTaskGroups\": 3, \"useConcurrentLocks\": true}}\n// then retry POST /druid/indexer/v1/supervisor/<id>/reset","handlingStrategy":"validation","validationCode":"const spec = await (await fetch(`${overlord}/druid/indexer/v1/supervisor/${id}`)).json();\nif (!spec.spec.context || spec.spec.context.useConcurrentLocks !== true) throw new Error('enable useConcurrentLocks before backfill');","typeGuard":"function hasConcurrentLocks(spec) { return spec?.spec?.context?.useConcurrentLocks === true; }","tryCatchPattern":"try { await resetBackfill(id); } catch (e) { if (/useConcurrentLocks/.test(e.message)) { await enableConcurrentLocks(id); await resetBackfill(id); } else throw e; }","preventionTips":["Set useConcurrentLocks=true in supervisor context when backfill workflows are planned","Standardize new supervisor specs on concurrent locks","Validate context flags as part of spec review"],"tags":["supervisor","configuration","locking","druid-overlord"],"backgroundTag":"missing-required-config-field","analyzedSha":"9b90983fd291f26935af934383ce360473179e4d","analyzedAt":"2026-09-07T13:32:30.957Z","contentChangedAt":"2026-09-07T13:32:30.957Z","schemaVersion":2},"datasetVersion":"2026-09-17T15:17:12.973Z"}