apache/druid · error · IllegalArgumentException

Cannot map region larger than %,d bytes

Error message

Cannot map region larger than %,d bytes

What it means

FileUtils.map(File, long, long) memory-maps a file region via FileChannel.map, but MappedByteBuffer is index-based and cannot exceed Integer.MAX_VALUE bytes. Any request for a larger region is rejected upfront with IllegalArgumentException.

Solutions

1. Map the file in chunks of at most Integer.MAX_VALUE bytes and process sequentially
2. Use FileChannel direct reads (ByteBuffer in a loop) instead of mmap for large regions
3. Use the long-typed reading APIs (e.g. FileUtils.map with wrapped handler loops or java.nio channels)
4. Reduce the requested offset/length to a region under 2GB

Example fix

// before
MappedByteBufferHandler h = FileUtils.map(file, 0, file.length());
// after
long remaining = file.length();
long offset = 0;
while (remaining > 0) {
  long chunk = Math.min(remaining, Integer.MAX_VALUE);
  try (MappedByteBufferHandler h = FileUtils.map(file, offset, chunk)) {
    process(h.getBuffer());
  }
  offset += chunk;
  remaining -= chunk;
}

if (length > Integer.MAX_VALUE) {
  throw new IllegalArgumentException("Use chunked mapping for " + length + " bytes");
}

try (MappedByteBufferHandler h = FileUtils.map(file, offset, length)) {
  process(h.getBuffer());
} catch (IllegalArgumentException e) {
  // fall back to chunked/stream reading
}

When it happens

Trigger: Calling FileUtils.map(file, offset, length) with length > Integer.MAX_VALUE (2,147,483,647 bytes).

Common situations: Mapping large segment files (multi-GB) whole; computing length from file size without a 2GB guard; legacy code written when files were small.

Understand the failure class

Background: "File too large" / "file size exceeds limit" errors: why libraries cap file sizes and how to fix them — this error's family across 46 libraries.

Related errors

• File mapping failed.
• Cannot add files of the same name, already have
• Cannot create directory
• Cannot have a comma in the name of a file, got
• Cannot list directory

AI-assisted analysis of apache/druid@9b90983fd2 (2026-09-07). Data as JSON: /api/errors/21468af9746524d8. Report an issue: GitHub.

Appendix: source

Thrown at processing/src/main/java/org/apache/druid/java/util/common/FileUtils.java:203

  /**
   * Fully maps a file read-only in to memory as per
   * {@link FileChannel#map(FileChannel.MapMode, long, long)}.
   *
   * @param file   the file to map
   * @param offset starting offset for the mmap
   * @param length length for the mmap
   *
   * @return a {@link MappedByteBufferHandler}, wrapping a read-only buffer reflecting {@code file}
   *
   * @throws FileNotFoundException    if the {@code file} does not exist
   * @throws IOException              if an I/O error occurs
   * @throws IllegalArgumentException if length is greater than {@link Integer#MAX_VALUE}
   * @see FileChannel#map(FileChannel.MapMode, long, long)
   */
  public static MappedByteBufferHandler map(File file, long offset, long length) throws IOException
  {
    if (length > Integer.MAX_VALUE) {
      throw new IAE("Cannot map region larger than %,d bytes", Integer.MAX_VALUE);
    }

    try (final RandomAccessFile randomAccessFile = new RandomAccessFile(file, "r");
         final FileChannel channel = randomAccessFile.getChannel()) {
      final MappedByteBuffer mappedByteBuffer = channel.map(FileChannel.MapMode.READ_ONLY, offset, length);
      return new MappedByteBufferHandler(mappedByteBuffer);
    }
  }

  /**
   * Fully maps a file read-only in to memory as per
   * {@link FileChannel#map(FileChannel.MapMode, long, long)}.
   *
   * @param randomAccessFile the file to map. The file will not be closed.
   * @param offset           starting offset for the mmap
   * @param length           length for the mmap
   *
   * @return a {@link MappedByteBufferHandler}, wrapping a read-only buffer reflecting {@code randomAccessFile}

View on GitHub (pinned to 9b90983fd2)