elastic/elasticsearch · error · IllegalArgumentException

expected an FSDirectory but got [{}] after unwrapping [{}]

Error message

expected an FSDirectory but got [{}] after unwrapping [{}]

What it means

Thrown by MemorySegmentUtils.unwrapFSDirectory() when the Lucene Directory, after unwrapping all FilterDirectory layers, is not an FSDirectory (filesystem-backed directory). The GPU codec needs a filesystem path to create file-backed MemorySegments for vector data when the data exceeds MMapDirectory's chunk size and must be manually mapped via FileChannel. RAMDirectory, ByteBuffersDirectory, or other non-filesystem directories cannot provide a real file path.

Source

Thrown at libs/gpu-codec/src/main/java/org/elasticsearch/gpu/codec/MemorySegmentUtils.java:64

        @Override
        default void close() {}
    }

    private MemorySegmentUtils() {}

    /**
     * Unwraps a {@link Directory} through any {@link FilterDirectory} layers to find the underlying {@link FSDirectory}.
     * Elasticsearch wraps directories (e.g. {@code Store$StoreDirectory} extends {@code FilterDirectory}), so a direct
     * cast to {@link FSDirectory} will fail at runtime.
     *
     * @throws IllegalArgumentException if the unwrapped directory is not an {@link FSDirectory}
     */
    static FSDirectory unwrapFSDirectory(Directory dir) {
        Directory unwrapped = FilterDirectory.unwrap(dir);
        if (unwrapped instanceof FSDirectory fsDir) {
            return fsDir;
        }
        throw new IllegalArgumentException(
            "expected an FSDirectory but got [" + unwrapped.getClass().getName() + "] after unwrapping [" + dir.getClass().getName() + "]"
        );
    }

    /**
     * Creates a file-backed MemorySegment, mapping the first {@param dataSize} bytes from {@param dataFile}, using the
     * Java {@link FileChannel} API.
     */
    static MemorySegmentHolder createFileBackedMemorySegment(Path dataFile, long dataSize) throws IOException {
        // Unwrap test-only filesystem layers so we get a real FileChannelImpl that supports Arena-based map.
        Path unwrappedPath = Unwrappable.unwrapAll(dataFile);
        Arena arena = null;
        try {
            arena = Arena.ofConfined();
            try (FileChannel fc = FileChannel.open(unwrappedPath, Set.of(READ))) {
                MemorySegment mapped = fc.map(FileChannel.MapMode.READ_ONLY, 0L, dataSize, arena);
                return new FileBackedMemorySegmentHolder(mapped, arena, dataFile);
            }

View on GitHub (pinned to db6a809a66)

Solutions

1. In tests, use FSDirectory (e.g. MMapDirectory or NIOFSDirectory) backed by a temp directory instead of RAMDirectory when testing the GPU codec with large vector segments.
2. Increase MMapDirectory's max chunk size so the data fits in a single mmap chunk, avoiding the fallback path entirely.
3. Ensure that test Directory wrappers properly extend FilterDirectory so unwrap() can find the underlying FSDirectory.
4. In production, verify that the index store configuration uses a filesystem-backed directory.

Example fix

// before — test with RAMDirectory
Directory dir = new RAMDirectory();

// after — test with FSDirectory
Path tmp = Files.createTempDirectory("gpu-test");
Directory dir = new MMapDirectory(tmp);

Directory unwrapped = FilterDirectory.unwrap(dir);
if (!(unwrapped instanceof FSDirectory)) {
    throw new IllegalStateException(
        "GPU codec requires FSDirectory for large vector segments, got: " + unwrapped.getClass().getName());
}

static boolean isFSDirectory(Directory dir) {
    return FilterDirectory.unwrap(dir) instanceof FSDirectory;
}

When it happens

Trigger: Calling getContiguousMemorySegment or getContiguousPackedMemorySegment with a Directory that is not backed by the filesystem. This happens when: the vector data is too large for a single mmap chunk (exceeds MMapDirectory.DEFAULT_MAX_CHUNK_SIZE, default 1GB on 64-bit JVMs), AND the Directory is a test/mock directory (RAMDirectory, NIOFSDirectory in test mode, etc.). In production, Elasticsearch always uses FSDirectory via Store.

Common situations: Unit/integration tests using RAMDirectory or MockDirectoryWrapper with vector data that exceeds the mmap chunk size; custom Directory implementation that doesn't extend FSDirectory; test fixtures that wrap directories in non-standard ways that prevent unwrapping to FSDirectory.

Related errors

• segment of size [{}] too small for expected {} float vectors
• Unable to create reaper JAR output directory {}
• Failed to write unicast_hosts for {}
• Can not start {}, is not a directory: {}
• Failed to create working directory for {}, with: {}

AI-assisted analysis of elastic/elasticsearch@db6a809a66 (2026-08-12). Data as JSON: /api/errors/e8f847268b90de28. Report an issue: GitHub.