{"record":{"id":"723b1ce8e17764fe","repo":"apache/druid","slug":"tasklogdir-s-must-be-a-directory","errorCode":null,"errorMessage":"taskLogDir [%s] must be a directory.","messagePattern":"taskLogDir \\[(.+?)\\] must be a directory\\.","errorType":"exception","errorClass":"IOE","httpStatus":null,"severity":"error","filePath":"extensions-core/hdfs-storage/src/main/java/org/apache/druid/storage/hdfs/tasklog/HdfsTaskLogs.java","lineNumber":191,"sourceCode":"  @Override\n  public void killAll() throws IOException\n  {\n    log.info(\"Deleting all task logs from hdfs dir [%s].\", config.getDirectory());\n    Path taskLogDir = new Path(config.getDirectory());\n    FileSystem fs = taskLogDir.getFileSystem(hadoopConfig);\n    fs.delete(taskLogDir, true);\n  }\n\n  @Override\n  public void killOlderThan(long timestamp) throws IOException\n  {\n    Path taskLogDir = new Path(config.getDirectory());\n    FileSystem fs = taskLogDir.getFileSystem(hadoopConfig);\n    if (fs.exists(taskLogDir)) {\n      FileStatus taskLogFileStatus = fs.getFileStatus(taskLogDir);\n\n      if (!taskLogFileStatus.isDirectory()) {\n        throw new IOE(\"taskLogDir [%s] must be a directory.\", taskLogDir);\n      }\n\n      RemoteIterator<LocatedFileStatus> iter = fs.listLocatedStatus(taskLogDir);\n      while (iter.hasNext()) {\n        LocatedFileStatus file = iter.next();\n        if (file.getModificationTime() < timestamp) {\n          Path p = file.getPath();\n          log.info(\"Deleting hdfs task log [%s].\", p.toUri().toString());\n          fs.delete(p, true);\n        }\n\n        if (Thread.currentThread().isInterrupted()) {\n          throw new IOException(\n              new InterruptedException(\"Thread interrupted. Couldn't delete all tasklogs.\")\n          );\n        }\n      }\n    }","sourceCodeStart":173,"sourceCodeEnd":209,"githubUrl":"https://github.com/apache/druid/blob/9b90983fd291f26935af934383ce360473179e4d/extensions-core/hdfs-storage/src/main/java/org/apache/druid/storage/hdfs/tasklog/HdfsTaskLogs.java#L173-L209","documentation":"HdfsTaskLogs.killOlderThan() validates that the configured task-log directory is a directory before iterating over old task logs. If the path exists in HDFS but is a regular file, an IOE is thrown. This prevents accidental deletion attempts against a wrongly configured path.","triggerScenarios":"killOlderThan(timestamp) finds fs.exists(taskLogDir) true but fs.getFileStatus(taskLogDir).isDirectory() false — i.e. config.getDirectory() points at a file.","commonSituations":"druid.indexer.logs.directory mistakenly set to a file path; someone overwrote/created a file at the directory path in HDFS; copy-paste typo in the task-log config.","solutions":["Fix druid.indexer.logs.directory in the middleManager/overlord config to point at a directory, e.g. hdfs://namenode/druid/tasklogs","Check what the path contains: hdfs dfs -ls <dir>; remove or rename the file if it was created by mistake","Restart the affected Druid services after correcting the config"],"exampleFix":"// before\ndruid.indexer.logs.directory=hdfs://nn/druid/tasklogs/current.log\n// after\ndruid.indexer.logs.directory=hdfs://nn/druid/tasklogs/","handlingStrategy":"validation","validationCode":"Path dir = new Path(config.getDirectory());\nFileSystem fs = dir.getFileSystem(hadoopConfig);\nif (!fs.exists(dir) || !fs.getFileStatus(dir).isDirectory()) {\n  throw new IllegalArgumentException(\"taskLogDir must be a directory: \" + dir);\n}","typeGuard":"null","tryCatchPattern":"try {\n  taskLogs.killOlderThan(cutoffMillis);\n} catch (IOException e) {\n  if (e.getMessage() != null && e.getMessage().startsWith(\"taskLogDir\")) {\n    // fix druid.indexer.logs.directory config\n  }\n  throw e;\n}","preventionTips":["Set druid.indexer.logs.directory to a directory path (trailing slash avoids typos)","After config changes, verify with hdfs dfs -ls that the path is a directory","Prevent accidental file creation at the configured path"],"tags":["hdfs","task-logs","configuration","path"],"backgroundTag":"path-is-not-a-directory","analyzedSha":"9b90983fd291f26935af934383ce360473179e4d","analyzedAt":"2026-09-07T13:32:30.957Z","contentChangedAt":"2026-09-07T13:32:30.957Z","schemaVersion":2},"datasetVersion":"2026-09-14T05:17:10.506Z"}