apache/druid · error · SegmentLoadingException

Couldn't delete segments from Azure. See the task logs for m

Error message

Couldn't delete segments from Azure. See the task logs for more details.

What it means

AzureDataSegmentKiller.kill aggregates per-batch segment deletions; if any batch or individual delete failed (batchSuccessful false or an exception was caught), it throws this generic SegmentLoadingException pointing the operator at task logs. The actual per-segment failure reason is logged, not carried in this exception.

Source

Thrown at extensions-core/azure-extensions/src/main/java/org/apache/druid/storage/azure/AzureDataSegmentKiller.java:107

      );
      keysToDelete.add(blobPath);
    }

    boolean shouldThrowException = false;
    for (Map.Entry<String, List<String>> containerToKeys : containerToKeysToDelete.entrySet()) {
      boolean batchSuccessful = azureStorage.batchDeleteFiles(
              containerToKeys.getKey(),
              containerToKeys.getValue(),
              null
      );

      if (!batchSuccessful) {
        shouldThrowException = true;
      }
    }

    if (shouldThrowException) {
      throw new SegmentLoadingException(
          "Couldn't delete segments from Azure. See the task logs for more details."
      );
    }
  }


  @Override
  public void kill(DataSegment segment) throws SegmentLoadingException
  {
    log.info("Killing segment [%s]", segment);

    Map<String, Object> loadSpec = segment.getLoadSpec();
    final String containerName = MapUtils.getString(loadSpec, "containerName");
    final String blobPath = MapUtils.getString(loadSpec, "blobPath");
    final String dirPath = Paths.get(blobPath).getParent().toString();

    try {
      azureStorage.emptyCloudBlobDirectory(containerName, dirPath);

View on GitHub (pinned to 9b90983fd2)

Solutions

Inspect the task/service logs for the per-segment BlobStorageException logged before this throw
Re-run the kill; failures are often transient or caused by already-deleted segments
Verify the Azure credentials have delete (blob write) permissions on the container
Check no concurrent kill tasks operate on the same segments

Example fix

// before
if (shouldThrowException) {
  throw new SegmentLoadingException("Couldn't delete segments from Azure. See the task logs for more details.");
}
// after
if (shouldThrowException) {
  throw new SegmentLoadingException(null, "Couldn't delete segments from Azure; last error: %s", lastErrorMessage);
}

Defensive patterns

Strategy: try-catch

Validate before calling

// verify permissions up front
boolean canDelete = testDeletePermission(container, prefix); // list + attempt small delete with a probe blob

Try / catch

try { killer.kill(segment); } catch (SegmentLoadingException e) { log.error("Azure kill failed, see cause logs: %s", e.getMessage()); /* retry later or alert */ }

Prevention

Grant delete permissions on the container to the Druid storage account
Avoid concurrent kill tasks on the same segments
Enable idempotent retry of kills; deletes are safe to re-run
Check task logs immediately — this exception deliberately omits the root cause

When it happens

Trigger: kill() or killBatch() fails to delete one or more segment blobs: emptyCloudBlobDirectory or deleteBlob throws (BlobStorageException, missing blob directory, permissions), or a batch reports partial failure.

Common situations: Killing segments whose files were already deleted (concurrent kill), storage account key lacking delete permission, or transient Azure 412/409 conflicts during batch kills.

Understand the failure class

Background: "API error: {status}" and "HTTP 401/403/404/429/5xx" errors: non-2xx HTTP responses explained — this error's family across 27 libraries.

Related errors

AI-assisted analysis of apache/druid@9b90983fd2 (2026-09-07). Data as JSON: /api/errors/24cb7884b23cb334. Report an issue: GitHub.