apache/druid · error · IllegalStateException (ISE)

Cannot delete all segment from S3 Deep Storage since druid.s

Error message

Cannot delete all segment from S3 Deep Storage since druid.storage.bucket and druid.storage.baseKey are not both set.

What it means

killAll() wipes every segment under druid.storage.baseKey, so it requires both druid.storage.bucket and druid.storage.baseKey to be configured; if either is null it throws this ISE rather than deleting from an unbounded location.

Source

Thrown at extensions-core/s3-extensions/src/main/java/org/apache/druid/storage/s3/S3DataSegmentKiller.java:216

        }
        // descriptor.json is a file to store segment metadata in deep storage. This file is deprecated and not stored
        // anymore, but we still delete them if exists.
        if (s3Client.doesObjectExist(s3Bucket, s3DescriptorPath)) {
          log.info("Removing descriptor file[s3://%s/%s] from s3!", s3Bucket, s3DescriptorPath);
          s3Client.deleteObject(s3Bucket, s3DescriptorPath);
        }
      }
    }
    catch (S3Exception e) {
      throw new SegmentLoadingException(e, "Couldn't kill segment[%s]: [%s]", segment.getId(), e);
    }
  }

  @Override
  public void killAll() throws IOException
  {
    if (segmentPusherConfig.getBucket() == null || segmentPusherConfig.getBaseKey() == null) {
      throw new ISE(
          "Cannot delete all segment from S3 Deep Storage since druid.storage.bucket and druid.storage.baseKey are not both set.");
    }
    log.info("Deleting all segment files from s3 location [bucket: '%s' prefix: '%s']",
             segmentPusherConfig.getBucket(), segmentPusherConfig.getBaseKey()
    );
    try {
      S3Utils.deleteObjectsInPath(
          s3ClientSupplier.get(),
          inputDataConfig.getMaxListingLength(),
          segmentPusherConfig.getBucket(),
          segmentPusherConfig.getBaseKey(),
          Predicates.alwaysTrue()
      );
    }
    catch (Exception e) {
      log.error("Error occurred while deleting segment files from s3. Error: %s", e.getMessage());
      throw new IOException(e);
    }

View on GitHub (pinned to 9b90983fd2)

Solutions

1. Set both druid.storage.bucket and druid.storage.baseKey in the runtime properties of the services that run kill tasks.
2. If properties are set via environment variables, confirm they are exported to the process and not shadowed by other config styles.
3. Only invoke killAll when full deep-storage wipe is intended; otherwise kill individual segments.

Example fix

// before (runtime.properties)
// (missing)
// after
druid.storage.bucket=my-druid-bucket
druid.storage.baseKey=druid/segments

// before invoking killAll, validate config
if (config.getBucket() == null || config.getBaseKey() == null) {
  throw new IllegalArgumentException("Set druid.storage.bucket and druid.storage.baseKey before killAll");
}

try { killer.killAll(); } catch (ISE e) { fixRuntimeProperties(); }

When it happens

Trigger: Calling the killAll supervision API (drop-all segments) when segmentPusherConfig.getBucket() or getBaseKey() is null, i.e. neither property is set in the coordinator/runtime config.

Common situations: Deployment uses s3-compatible config properties (env vars) instead of druid.storage.bucket/baseKey, or killAll invoked in a setup where only a specific bucket config exists.

Understand the failure class

Background: "is required", "must be set", "missing required field": configuration validation errors across open-source libraries — this error's family across 36 libraries.

Related errors

• Set 'account' to the storage account that needs to be config
• Target S3 bucket is not specified
• Target S3 baseKey is not specified
• valid values for maxListingLength are between [%d, %d]
• Invalid druid.storage.transfer.asyncHttpClientType[%s]. Must

AI-assisted analysis of apache/druid@9b90983fd2 (2026-09-07). Data as JSON: /api/errors/3163f41d64ceb45e. Report an issue: GitHub.