apache/beam · error · IllegalStateException

Scheme: [%s] has conflicting filesystems: [%s]

Error message

Scheme: [%s] has conflicting filesystems: [%s]

What it means

FileSystems.verifySchemesAreUnique detects two or more different FileSystem implementations registered for the same URL scheme and throws IllegalStateException listing the conflicting class names. This prevents ambiguous filesystem routing when multiple providers claim one scheme.

Source

Thrown at sdks/java/core/src/main/java/org/apache/beam/sdk/io/FileSystems.java:640

  static Map<String, FileSystem> verifySchemesAreUnique(
      PipelineOptions options, Set<FileSystemRegistrar> registrars) {
    Multimap<String, FileSystem> fileSystemsBySchemes =
        TreeMultimap.create(Ordering.<String>natural(), Ordering.arbitrary());

    for (FileSystemRegistrar registrar : registrars) {
      for (FileSystem fileSystem : registrar.fromOptions(options)) {
        fileSystemsBySchemes.put(fileSystem.getScheme(), fileSystem);
      }
    }
    for (Entry<String, Collection<FileSystem>> entry : fileSystemsBySchemes.asMap().entrySet()) {
      if (entry.getValue().size() > 1) {
        String conflictingFileSystems =
            Joiner.on(", ")
                .join(
                    FluentIterable.from(entry.getValue())
                        .transform(input -> input.getClass().getName())
                        .toSortedList(Ordering.natural()));
        throw new IllegalStateException(
            String.format(
                "Scheme: [%s] has conflicting filesystems: [%s]",
                entry.getKey(), conflictingFileSystems));
      }
    }

    ImmutableMap.Builder<String, FileSystem> schemeToFileSystem = ImmutableMap.builder();
    for (Entry<String, FileSystem> entry : fileSystemsBySchemes.entries()) {
      schemeToFileSystem.put(entry.getKey(), entry.getValue());
    }
    return schemeToFileSystem.build();
  }

  /**
   * Returns a new {@link ResourceId} that represents the named resource of a type corresponding to
   * the resource type.
   *
   * <p>The supplied {@code singleResourceSpec} is expected to be in a proper format, including any

View on GitHub (pinned to 12126d8942)

Solutions

  1. Run mvn dependency:tree / gradle dependencies and exclude duplicate Beam IO modules providing the same scheme
  2. Align all org.apache.beam artifacts to one version
  3. Check META-INF/services/org.apache.beam.software... FileSystemRegistrar registrations inside bundled jars and remove the conflicting jar
  4. In worker images, clean up stale Beam jars (e.g. in Spark/Flink lib dirs)

Example fix

// before (pom.xml)
<dependency>org.apache.beam:beam-sdks-java-io-google-cloud-platform:2.40.0</dependency>
<dependency>org.apache.beam:beam-sdks-java-io-google-cloud-platform:2.50.0</dependency>
// after
<dependency>org.apache.beam:beam-sdks-java-io-google-cloud-platform:2.50.0</dependency>
<!-- single version, duplicates excluded -->
Defensive patterns

Strategy: validation

Validate before calling

// Check the classpath for duplicate Beam IO jars before launching
Set<String> seen = new HashSet<>();
for (FileSystemRegistrar r : ServiceLoader.load(FileSystemRegistrar.class)) {
  for (String s : r.schemes()) {
    if (!seen.add(s)) throw new IllegalStateException("Duplicate scheme: " + s);
  }
}

Try / catch

try {
  FileSystems.setDefaultPipelineOptions(options);
} catch (IllegalStateException e) {
  if (e.getMessage().contains("has conflicting filesystems")) {
    throw new IllegalStateException("Duplicate Beam IO jars on classpath; run dependency:tree", e);
  }
  throw e;
}

Prevention

When it happens

Trigger: Setting default pipeline options (FileSystems.setDefaultPipelineOptions) when the classpath contains multiple FileSystemRegistrar service-loader entries producing filesystems with the same scheme — typically duplicate or incompatible Beam IO jars on the classpath.

Common situations: Shading multiple Beam versions into one fat jar; including both a legacy and new filesystem provider for the same scheme; classpath pollution in Spark/Flink/Beam worker images.

Understand the failure class

Background: Conflicting config options: "cannot be used together" — configuration validation errors across open-source libraries — this error's family across 162 libraries.

Related errors

AI-assisted analysis of apache/beam@12126d8942 (2026-09-13). Data as JSON: /api/errors/33d2060d113d2602. Report an issue: GitHub.