What causes "Cannot add files of the same name, already have [%s]" in apache/druid?

Make each name unique before adding, e.g. append an index/counter or partition id: name + "_" + i.

apache/druid · error · IllegalArgumentException

Cannot add files of the same name, already have

Error message

Cannot add files of the same name, already have [%s]

What it means

A FileSmoosher container keys its internal files by name in a map, so each name must be unique within one smoosh output. add() throws this IllegalArgumentException when you try to add a second buffer under a name that was already added to this FileSmoosher.

Solutions

Make each name unique before adding, e.g. append an index/counter or partition id: name + "_" + i.
Check internalFiles contents (or track names yourself) before calling add and skip/rename duplicates.
If the duplicate indicates a logic bug, open a fresh FileSmoosher per logical unit instead of reusing one.
Deduplicate input entries before the write loop.

Example fix

// before
for (ByteBuffer buf : buffers) {
  smoosher.add("data", buf); // duplicate name
}
// after
for (int i = 0; i < buffers.size(); i++) {
  smoosher.add("data_" + i, buffers.get(i));
}

Defensive patterns

Strategy: validation

Validate before calling

java.util.Set<String> seen = new java.util.HashSet<>();
if (!seen.add(name)) {
  name = name + "_" + seen.size(); // or skip
}
smoosher.add(name, buffer);

Try / catch

try {
  smoosher.add(name, buffer);
} catch (IllegalArgumentException e) {
  if (e.getMessage() != null && e.getMessage().startsWith("Cannot add files of the same name")) {
    smoosher.add(name + "_dup" + counter.incrementAndGet(), buffer);
  } else { throw e; }
}

Prevention

Generate names with a unique suffix (partition id, counter).
Track added names in a local Set before adding.
Don't reuse a FileSmoosher across logically separate write batches.

When it happens

Trigger: Calling add(String, ByteBuffer) or addWithChannel(String, ...) twice with the same name on the same FileSmoosher instance before closing it.

Common situations: Re-running an add loop without changing the generated name (missing partition/counter suffix); retrying an add after a partial failure without opening a new FileSmoosher; merging data sources that produce identically named files.

Understand the failure class

Background: "already exists" / EEXIST / FileAlreadyExistsException: what the 'file already exists' error means and how to fix it — this error's family across 37 libraries.

Related errors

AI-assisted analysis of apache/druid@9b90983fd2 (2026-09-07). Data as JSON: /api/errors/6ae0ee7f4da3f621. Report an issue: GitHub.

Appendix: source

Thrown at processing/src/main/java/org/apache/druid/java/util/common/io/smoosh/FileSmoosher.java:150

  }

  @Override
  public void add(String name, File fileToAdd) throws IOException
  {
    try (MappedByteBufferHandler fileMappingHandler = FileUtils.map(fileToAdd)) {
      add(name, fileMappingHandler.get());
    }
  }

  @Override
  public void add(String name, ByteBuffer bufferToAdd) throws IOException
  {
    if (name.contains(",")) {
      throw new IAE("Cannot have a comma in the name of a file, got[%s].", name);
    }

    if (internalFiles.get(name) != null) {
      throw new IAE("Cannot add files of the same name, already have [%s]", name);
    }

    long size = 0;
    size += bufferToAdd.remaining();

    try (SegmentFileChannel out = addWithChannel(name, size)) {
      out.write(bufferToAdd);
    }
  }

  @Override
  public SegmentFileChannel addWithChannel(final String name, final long size) throws IOException
  {
    return addWithSmooshedWriter(name, size);
  }

  @Override
  public void abort()

View on GitHub (pinned to 9b90983fd2)