What causes "taskLogDir [%s] must be a directory." in apache/druid?

Inspect config.getDirectory() path on the affected node and remove or rename the non-directory entry, then recreate it as a directory: mkdir -p

apache/druid · error · IOException

taskLogDir [%s] must be a directory.

Error message

taskLogDir [%s] must be a directory.

What it means

FileTaskLogs.killOlderThan deletes local task log files older than the given timestamp; before listing, it validates that the configured taskLogDir exists and is a directory. This IOE is thrown when config.getDirectory() points to an existing path that is a regular file (or other non-directory), so log cleanup cannot proceed. It indicates a bad druid.indexer.logs.directory configuration rather than a transient failure.

Source

Thrown at indexing-service/src/main/java/org/apache/druid/indexing/common/tasklogs/FileTaskLogs.java:129

  {
    return new File(config.getDirectory(), StringUtils.format("%s.%s", taskid, filename));
  }

  @Override
  public void killAll() throws IOException
  {
    log.info("Deleting all task logs from local dir [%s].", config.getDirectory().getAbsolutePath());
    FileUtils.deleteDirectory(config.getDirectory());
  }

  @Override
  public void killOlderThan(final long timestamp) throws IOException
  {
    File taskLogDir = config.getDirectory();
    if (taskLogDir.exists()) {

      if (!taskLogDir.isDirectory()) {
        throw new IOE("taskLogDir [%s] must be a directory.", taskLogDir);
      }

      File[] files = taskLogDir.listFiles(f -> f.lastModified() < timestamp);

      for (File file : files) {
        log.info("Deleting local task log [%s].", file.getAbsolutePath());
        org.apache.commons.io.FileUtils.forceDelete(file);

        if (Thread.currentThread().isInterrupted()) {
          throw new IOException(
              new InterruptedException("Thread interrupted. Couldn't delete all tasklogs.")
          );
        }
      }
    }
  }
}

View on GitHub (pinned to 9b90983fd2)

Solutions

Inspect config.getDirectory() path on the affected node and remove or rename the non-directory entry, then recreate it as a directory: mkdir -p <path>
Correct druid.indexer.logs.directory in the node's runtime.properties to point to a proper directory and restart the service
Check volume/mount configuration if running in containers or on NFS
Optionally pre-validate with a startup check (Files.isDirectory) to fail fast at boot

Example fix

// before: only checked at killOlderThan time
File taskLogDir = config.getDirectory();
if (!taskLogDir.isDirectory()) { throw new IOE(...); }
// after: fail fast at startup
PreConditions.checkArgument(Files.isDirectory(config.getDirectory().toPath()), "taskLogDir must be a directory");
// or on the host: rm badpath && mkdir -p badpath

Defensive patterns

Strategy: validation

Validate before calling

// pre-validate the configured log directory on service startup
File dir = config.getDirectory();
if (dir.exists() && !dir.isDirectory()) {
  throw new IllegalStateException("druid.indexer.logs.directory must be a directory: " + dir);
}

Type guard

static boolean isUsableLogDir(File dir) {
  return dir != null && (!dir.exists() || dir.isDirectory());
}

Try / catch

catch (IOException e) {
  if (e.getMessage() != null && e.getMessage().startsWith("taskLogDir")) {
    log.error("fix druid.indexer.logs.directory (not a directory); skipping log cleanup this cycle");
    return; // don't crash retention loop; alert instead
  } throw e;
}

Prevention

Validate druid.indexer.logs.directory in startup health checks
Ensure container/volume mounts map to real directories, not files
Don't create files at the log directory path (watch for stray logs/redirects)
Verify the path on every node individually in multi-node clusters

When it happens

Trigger: druid.indexer.logs.directory (or equivalent FileTaskLogsConfig directory) resolves to an existing file, symlink to a file, or mount point that is not a directory when the middle manager/overlord runs killOlderThan for log retention.

Common situations: Log directory accidentally replaced by a file (e.g., a stray mount, a file created at the same path); misconfigured or typo'd path colliding with an existing file; container volume mounted incorrectly; permissions changes that left a file where the directory used to be.

Related errors

AI-assisted analysis of apache/druid@9b90983fd2 (2026-09-07). Data as JSON: /api/errors/752f0ee211f1bf92. Report an issue: GitHub.