apache/flink · error · IOException

cache file {name}already exists!

Error message

cache file {name}already exists!

What it means

Thrown by Plan.registerCachedFile when a cache file is registered twice under the same name. The Plan keeps a Map of distributed-cache entries keyed by name; registering a duplicate would silently overwrite the first file, so the API rejects it with an IOException. Note the message has a missing space ('name + "already exists!"') but the cause is unambiguous.

Source

Thrown at flink-core/src/main/java/org/apache/flink/api/common/Plan.java:342

    @Override
    public void accept(Visitor<Operator<?>> visitor) {
        for (GenericDataSinkBase<?> sink : this.sinks) {
            sink.accept(visitor);
        }
    }

    /**
     * Register cache files at program level.
     *
     * @param entry contains all relevant information
     * @param name user defined name of that file
     * @throws java.io.IOException
     */
    public void registerCachedFile(String name, DistributedCacheEntry entry) throws IOException {
        if (!this.cacheFile.containsKey(name)) {
            this.cacheFile.put(name, entry);
        } else {
            throw new IOException("cache file " + name + "already exists!");
        }
    }

    /**
     * Return the registered cached files.
     *
     * @return Set of (name, filePath) pairs
     */
    public Set<Entry<String, DistributedCacheEntry>> getCachedFiles() {
        return this.cacheFile.entrySet();
    }

    public int getMaximumParallelism() {
        MaxDopVisitor visitor = new MaxDopVisitor();
        accept(visitor);
        return Math.max(visitor.maxDop, this.defaultParallelism);
    }

View on GitHub (pinned to 2f3c205e92)

Solutions

1. Guard with getCachedFiles()/contains check before registering, or use a Set of names already added.
2. Centralize cache-file registration in one place in your program setup to avoid double-registration across operators.
3. If a file may legitimately be reused, register once at program assembly time and reference the name from all operators.

Example fix

// before
plan.registerCachedFile("model", entry1);
plan.registerCachedFile("model", entry2);
// after
Set<String> names = plan.getCachedFiles().stream()
    .map(Map.Entry::getKey).collect(Collectors.toSet());
if (!names.contains("model")) {
    plan.registerCachedFile("model", entry1);
}

Set<String> names = plan.getCachedFiles().stream()
    .map(Map.Entry::getKey).collect(Collectors.toSet());
if (!names.contains(name)) {
    plan.registerCachedFile(name, entry);
}

try { plan.registerCachedFile(name, entry); }
catch (IOException e) { /* duplicate; ignore or update */ }

When it happens

Trigger: Calling registerCachedFile(name, entry) twice with the same name on one Plan; iterating a config that lists the same file under one name; a library that auto-registers a file a user also registered manually.

Common situations: Registering the same distributed-cache resource (e.g., a model file, lookup table) in multiple operator setup methods; merging configs from sub-modules that both register 'weights.bin'.

Related errors

• An aggregator is already registered under the given name.
• File with name '${name}' is not available. Did you forget to
• A TypeInfoFactory for type '{}' is already registered.
• Failed to get the FileSystem of artifact {artifactFilePath}.
• Execution config has not been set properly for this plan

AI-assisted analysis of apache/flink@2f3c205e92 (2026-08-14). Data as JSON: /api/errors/f70f0873ad0b12c2. Report an issue: GitHub.