apache/beam · error · RuntimeException

Error determining if

Error message

Error determining if %s allows dynamic splitting

What it means

FileBasedReader.allowsDynamicSplitting() wraps any exception from getCurrentSource().isSplittable() into a RuntimeException 'Error determining if %s allows dynamic splitting'. It indicates the reader could not query its source's splittability — usually a filesystem/IO failure while resolving the current source.

Solutions

1. Inspect the chained cause for the underlying IO error and fix storage connectivity.
2. Retry the pipeline; transient storage failures often resolve.
3. If you implemented a custom FileBasedSource, make isSplittable() not throw.
4. Disable/avoid dynamic splitting scenarios by reading bounded, well-formed file patterns.

try { reader.allowsDynamicSplitting(); } catch (RuntimeException e) { log.warn("Splittability check failed for {}", e.getCause(), e); }

When it happens

Trigger: isSplittable() throwing during dynamic work rebalancing — underlying filesystem errors (metadata lookup failing), source in a bad state after read errors, or runner-driven split attempts during IO instability.

Common situations: Runner attempts dynamic splitting while the backing file/pattern is temporarily unavailable (GCS 5xx, HDFS namenode issues), or custom FileBasedSource.isSplittable() implementations that throw.

Understand the failure class

Background: "failed to read file", EACCES, ENOENT and "could not read <path>" errors: when a program can't read a file from disk — this error's family across 49 libraries.

Related errors

• Cannot estimate the row count. All the sampled lines are…
• CheckpointMark must override getOffsetLimit() if…
• Error completing file copies with retries, sample: from
• Error matching file spec
• Error trying to delete

AI-assisted analysis of apache/beam@12126d8942 (2026-09-13). Data as JSON: /api/errors/0fa9667754a6df03. Report an issue: GitHub.

Appendix: source

Thrown at sdks/java/core/src/main/java/org/apache/beam/sdk/io/FileBasedSource.java:562

    /**
     * Closes any {@link ReadableByteChannel} created for the current reader. This implementation is
     * idempotent. Any {@code close()} method introduced by a subclass must be idempotent and must
     * call the {@code close()} method in the {@code FileBasedReader}.
     */
    @Override
    public void close() throws IOException {
      if (channel != null) {
        channel.close();
      }
    }

    @Override
    public boolean allowsDynamicSplitting() {
      try {
        return getCurrentSource().isSplittable();
      } catch (Exception e) {
        throw new RuntimeException(
            String.format("Error determining if %s allows dynamic splitting", this), e);
      }
    }

    /**
     * Performs any initialization of the subclass of {@code FileBasedReader} that involves IO
     * operations. Will only be invoked once and before that invocation the base class will seek the
     * channel to the source's starting offset.
     *
     * <p>Provided {@link ReadableByteChannel} is for the file represented by the source of this
     * reader. Subclass may use the {@code channel} to build a higher level IO abstraction, e.g., a
     * BufferedReader or an XML parser.
     *
     * <p>If the corresponding source is for a subrange of a file, {@code channel} is guaranteed to
     * be an instance of the type {@link SeekableByteChannel}.
     *
     * <p>After this method is invoked the base class will not be reading data from the channel or
     * adjusting the position of the channel. But the base class is responsible for properly closing

View on GitHub (pinned to 12126d8942)