apache/beam · error · IllegalArgumentException

Sorter doesn't support GCS temporary location.

Error message

Sorter doesn't support GCS temporary location.

What it means

Beam's ExternalSorter (the Hadoop-based native sorter in the sorter extension) spills intermediate sorted data to a local temporary directory and cannot read/write paths on Google Cloud Storage. Options.setTempLocation rejects any path starting with gs:// with an IllegalArgumentException, because the sorter requires random-access local filesystem access that GCS does not provide.

Source

Thrown at sdks/java/extensions/sorter/src/main/java/org/apache/beam/sdk/extensions/sorter/ExternalSorter.java:43

public abstract class ExternalSorter implements Sorter {
  protected final Options options;

  /** {@link Options} contains configuration of the sorter. */
  public static class Options implements Serializable {
    private String tempLocation = "/tmp";
    private int memoryMB = 100;
    private SorterType sorterType = SorterType.HADOOP;

    /** Sorter type. */
    public enum SorterType {
      HADOOP,
      NATIVE
    }

    /** Sets the path to a temporary location where the sorter writes intermediate files. */
    public Options setTempLocation(String tempLocation) {
      if (tempLocation.startsWith("gs://")) {
        throw new IllegalArgumentException("Sorter doesn't support GCS temporary location.");
      }

      this.tempLocation = tempLocation;
      return this;
    }

    /** Returns the configured temporary location. */
    public String getTempLocation() {
      return tempLocation;
    }

    /**
     * Sets the size of the memory buffer in megabytes. Must be greater than zero and less than
     * 2048.
     */
    public Options setMemoryMB(int memoryMB) {
      this.memoryMB = memoryMB;
      checkMemoryMB();

View on GitHub (pinned to 12126d8942)

Solutions

1. Set a local (non-GCS) temp location: options.setTempLocation("/tmp/sorter") or another local path on the worker
2. If on Dataflow, derive a worker-local directory (e.g., use the worker's /tmp) instead of forwarding the pipeline's gs:// tempLocation
3. Switch to a sorter implementation that supports GCS if remote temp storage is required, or stage via a different mechanism (e.g., write sorted output yourself)
4. Guard the configuration: strip or validate the tempLocation scheme before calling setTempLocation

Example fix

// before
new Options().setTempLocation(pipelineOptions.getTempLocation()); // "gs://bucket/tmp"
// after
new Options().setTempLocation(Files.createTempDirectory("sorter").toString());

if (tempLocation != null && tempLocation.startsWith("gs://")) { throw new IllegalArgumentException("Provide a local temp dir for Sorter"); }
options.setTempLocation(tempLocation);

static boolean isLocalPath(String loc) { return loc != null && !loc.startsWith("gs://") && !loc.startsWith("s3://") && !loc.startsWith("abfs://"); }

try { options.setTempLocation(tempLocation); } catch (IllegalArgumentException e) { options.setTempLocation(Files.createTempDirectory("sorter").toString()); }

When it happens

Trigger: Calling ExternalSorter.create(...) or configuring its Options with setTempLocation("gs://bucket/path") — e.g., when --tempLocation is inherited directly from a Dataflow pipeline option that is a GCS path.

Common situations: Running on Dataflow where the pipeline's --tempLocation is gs:// and is passed straight through to the sorter options; building sorter options programmatically from pipeline temp location defaults.

Understand the failure class

Background: UnsupportedOperationException and "is not supported" errors: when a library deliberately refuses a call — this error's family across 30 libraries.

Related errors

• No filesystem found for scheme
• Exploded field %s must be an iterable type, got %s.
• boolean cross product parameter required to explode more tha
• ${config}
• Unknown log level ${level}. Valid log levels are ${validLeve

AI-assisted analysis of apache/beam@12126d8942 (2026-09-13). Data as JSON: /api/errors/5d2953d68bb115ff. Report an issue: GitHub.