Error while cleaning up temporary files at path[%s]. Skippin

Error message

Error while cleaning up temporary files at path[%s]. Skipping.

What it means

At the end of a MSQ controller run, ControllerImpl deletes the controller's temporary directory through the storage connector as a failsafe cleanup. If deleteRecursively throws, the controller logs 'Error while cleaning up temporary files at path[%s]. Skipping.' and continues — cleanup errors never fail the query.

Source

Thrown at multi-stage-query/src/main/java/org/apache/druid/msq/exec/ControllerImpl.java:1984

  /**
   * Clean up durable storage, if used for stage output.
   * <p>
   * Note that this is only called by the controller task itself. It isn't called automatically by anything in
   * particular if the controller fails early without being able to run its cleanup routines. This can cause files
   * to be left in durable storage beyond their useful life.
   */
  private void cleanUpDurableStorageIfNeeded()
  {
    if (queryKernelConfig != null && queryKernelConfig.isDurableStorage()) {
      final String controllerDirName = DurableStorageUtils.getControllerDirectory(queryId());
      try {
        // Delete all temporary files as a failsafe
        MSQTasks.makeStorageConnector(context.injector()).deleteRecursively(controllerDirName);
      }
      catch (Exception e) {
        // If an error is thrown while cleaning up a file, log it and try to continue with the cleanup
        log.warn(e, "Error while cleaning up temporary files at path[%s]. Skipping.", controllerDirName);
      }
    }
  }

  private static String getDataSourceForIngestion(final MSQSpec querySpec)
  {
    return ((DataSourceMSQDestination) querySpec.getDestination()).getDataSource();
  }

  /**
   * Compute shard columns for {@link DimensionRangeShardSpec}. Returns an empty list if range-based sharding
   * is not applicable.
   *
   * @return pair of shard columns and commentary
   */
  private static Pair<List<String>, String> computeShardColumns(
      final RowSignature signature,
      final ClusterBy clusterBy,

View on GitHub (pinned to 9b90983fd2)

Solutions

Check the attached exception and manually delete the reported controllerDirName path if it remains.
Verify permissions and free space on the base task directory / storage location used by the MSQ storage connector.
Retry or rely on periodic garbage collection (e.g. druid-cleaner for the local task dir) to remove stale temporary files.

Defensive patterns

Strategy: try-catch

Try / catch

try {
  connector.deleteRecursively(dirName);
} catch (Exception e) {
  log.warn(e, "Temp cleanup failed for %s; schedule manual/GC removal", dirName);
}

Prevention

Ensure correct permissions and free space on the base task directory.
Enable/verify the Druid cleaner job to garbage-collect leftover temp dirs.
Watch cloud storage rate limits (S3 503s) during recursive deletes.

When it happens

Trigger: The final cleanup call MSQTasks.makeStorageConnector(context.injector()).deleteRecursively(controllerDirName) throws, e.g. due to local disk errors, permissions on the job temp directory, or storage connector (S3/HDFS/local) transient failures.

Common situations: Permission problems on druid.indexer.task.baseTaskDir, disk-full or NFS/HDFS flakiness, or cloud storage throttling (S3 503s) during recursive deletion; leftover temp files may accumulate on the affected storage.

Related errors

AI-assisted analysis of apache/druid@9b90983fd2 (2026-09-07). Data as JSON: /api/errors/8711cb877b27bdf2. Report an issue: GitHub.