apache/druid · error · SegmentLoadingException

Unable to kill segment

Error message

Unable to kill segment

What it means

The kill operation wraps any IOException encountered while deleting local segment files into a SegmentLoadingException with the message "Unable to kill segment", preserving the cause. It signals filesystem-level failure (permissions, missing parent directories, I/O errors) during segment cleanup.

Source

Thrown at server/src/main/java/org/apache/druid/segment/loading/LocalDataSegmentKiller.java:85

        int maxDepth = 4; // if for some reason there's no datasSource directory, stop recursing somewhere reasonable
        while (parentDir != null && --maxDepth >= 0) {
          // parentDir.listFiles().length > 0 check not strictly necessary, because parentDir.delete() fails on
          // nonempty directories. However, including it here is nice since it makes our intent very clear (only
          // remove nonempty directories) and it prevents making delete syscalls that are doomed to failure.
          if (parentDir.listFiles().length > 0
              || !parentDir.delete()
              || segment.getDataSource().equals(parentDir.getName())) {
            break;
          }

          parentDir = parentDir.getParentFile();
        }
      } else if (path.exists()) {
        throw new SegmentLoadingException("Unknown file type[%s]", path);
      }
    }
    catch (IOException e) {
      throw new SegmentLoadingException(e, "Unable to kill segment");
    }
  }

  @Override
  public void killAll() throws IOException
  {
    log.info("Deleting all segments from directory[%s].", storageDirectory.getAbsolutePath());
    FileUtils.deleteDirectory(storageDirectory);
  }

  private File getPath(DataSegment segment)
  {
    return new File(MapUtils.getString(segment.getLoadSpec(), PATH_KEY));
  }
}

View on GitHub (pinned to 9b90983fd2)

Solutions

1. Check filesystem permissions on the segment cache directory (ls -la) and grant write access to the Druid process user.
2. Look at the wrapped IOException cause in the exception stack trace for the exact filesystem error.
3. Ensure the volume hosting the segment directory is mounted read-write and healthy, then retry the kill.

Example fix

// before
$ ls -ld /var/druid/segments  # root-owned
// after
$ chown -R druid:druid /var/druid/segments && retry kill via coordinator cleanup

File dir = new File(segmentDir);
boolean canWrite = dir.exists() && dir.canWrite();

try {
  killer.kill(segment);
} catch (SegmentLoadingException e) {
  if (e.getCause() instanceof IOException) {
    log.error(e.getCause(), "Filesystem failure killing %s: check permissions/mount", segment.getId());
  }
}

When it happens

Trigger: Calling kill or killAll when the local segment directory cannot be read/deleted — e.g. permission changes, read-only mounts, or concurrent deletion between the exists() check and delete.

Common situations: Running Druid as a user without ownership of the historical segment cache; NFS/EBS volume failures; administrators manually chmod'ing druid.storage dir.

Understand the failure class

Background: "failed to write file", "Could not save figure", "Error saving remote file" — file write failed: causes and fixes across languages and libraries — this error's family across 38 libraries.

Related errors

• Cannot create tempDir [%s] for google storage connector
• Could not create temp distribution directory.
• Path [%s] is not writable, check permissions
• No files found in [%s]
• Unauthorized

AI-assisted analysis of apache/druid@9b90983fd2 (2026-09-07). Data as JSON: /api/errors/2686a33b0759fe09. Report an issue: GitHub.