elastic/elasticsearch · error · IOException

could not remove the following files (in the order of attemp

Error message

could not remove the following files (in the order of attempts):
   {}: {}
...

What it means

Thrown by IOUtils.rm after it has attempted to delete every file and directory under the given paths. rm collects per-path failures into a LinkedHashMap (path -> Throwable) and, if any remain, raises a single IOException listing each failed path with its cause. The multi-line message preserves attempt order so the most common root cause (usually permissions or a held file handle) is visible first.

Source

Thrown at libs/core/src/main/java/org/elasticsearch/core/IOUtils.java:203

                }
            }
        }
    }

    /**
     * Deletes one or more files or directories (and everything underneath it).
     *
     * @throws IOException if any of the given files (or their sub-hierarchy files in case of directories) cannot be removed.
     */
    public static void rm(final Path... locations) throws IOException {
        final LinkedHashMap<Path, Throwable> unremoved = rm(new LinkedHashMap<>(), locations);
        if (unremoved.isEmpty() == false) {
            final StringBuilder b = new StringBuilder("could not remove the following files (in the order of attempts):\n");
            for (final Map.Entry<Path, Throwable> kv : unremoved.entrySet()) {
                b.append("   ").append(kv.getKey().toAbsolutePath()).append(": ").append(kv.getValue()).append("\n");
            }
            throw new IOException(b.toString());
        }
    }

    private static LinkedHashMap<Path, Throwable> rm(final LinkedHashMap<Path, Throwable> unremoved, final Path... locations) {
        if (locations != null) {
            for (final Path location : locations) {
                // TODO: remove this leniency
                if (location != null && Files.exists(location)) {
                    try {
                        Files.walkFileTree(location, new FileVisitor<Path>() {
                            @Override
                            public FileVisitResult preVisitDirectory(final Path dir, final BasicFileAttributes attrs) throws IOException {
                                return FileVisitResult.CONTINUE;
                            }

                            @Override
                            public FileVisitResult postVisitDirectory(final Path dir, final IOException impossible) throws IOException {
                                assert impossible == null;

View on GitHub (pinned to db6a809a66)

Solutions

  1. Close any IndexReader, Directory, or channel that may hold the path before calling rm (use try-with-resources).
  2. Read the listed cause for the first failing path — it typically identifies the real problem (access denied, file in use).
  3. On Windows, ensure no mmap'd segment is still mapped; prefer IOUtils.FS.setMmapEnabled(false) in tests that delete aggressively.
  4. Check filesystem permissions and ownership of the listed paths; run as the owner or fix chmod/chown.
  5. If partial deletion is acceptable, wrap rm in a try and fall back to best-effort cleanup, but treat the failure as a leak signal.

Example fix

// before
IOUtils.rm(indexDir); // throws — reader still open

// after — close readers first
try (Directory dir = FSDirectory.open(indexDir);
     IndexReader r = DirectoryReader.open(dir)) {
    // use r
}
IOUtils.rm(indexDir);
Defensive patterns

Strategy: try-catch

Validate before calling

// Best-effort pre-check: confirm no locks/permissions issues before rm
void safeRm(Path p) throws IOException {
    if (!Files.exists(p)) return;
    try {
        IOUtils.rm(p);
    } catch (IOException e) {
        // log details and rethrow or degrade
    }
}

Type guard

static boolean isLikelyRemovable(Path p) {
    try {
        return Files.isWritable(p) && Files.exists(p);
    } catch (SecurityException e) {
        return false;
    }
}

Try / catch

try {
    IOUtils.rm(dir);
} catch (IOException e) {
    // parse the per-path causes; if access denied, fix permissions and retry;
    // if file in use, close handles and retry once. Otherwise surface the failure.
    log.warn("rm failed: {}", e.getMessage());
    throw e;
}

Prevention

When it happens

Trigger: Calling IOUtils.rm(path...) on a directory tree where one or more files cannot be deleted: open file handles (mmap, Lucene index reader), permission denied, read-only filesystem, or a path concurrently modified by another process. The IOException is thrown only after the full walk completes.

Common situations: Test teardown deleting a temp index directory while an IndexReader/Directory is still open. Snapshots or recovery holding handles. Windows notoriously holds files open. Container/CI runs with an unprivileged user against a root-owned path. NFS/network filesystem locking.

Related errors

AI-assisted analysis of elastic/elasticsearch@db6a809a66 (2026-08-12). Data as JSON: /api/errors/4a2b979b7ad5c554. Report an issue: GitHub.