apache/beam · error · RuntimeException

Failed to determine if the source is splittable

Error message

Failed to determine if the source is splittable

What it means

The CompressedSource constructor calls isSplittable() on the delegate source to decide whether offsets can be split; if that check throws for any reason, the exception is wrapped in a RuntimeException 'Failed to determine if the source is splittable'. This is an internal precondition failure, usually caused by the delegate's isSplittable() implementation throwing.

Source

Thrown at sdks/java/core/src/main/java/org/apache/beam/sdk/io/CompressedSource.java:232

  /**
   * Creates a {@code CompressedSource} for an individual file. Used by {@link
   * CompressedSource#createForSubrangeOfFile}.
   */
  private CompressedSource(
      FileBasedSource<T> sourceDelegate,
      DecompressingChannelFactory channelFactory,
      Metadata metadata,
      long minBundleSize,
      long startOffset,
      long endOffset) {
    super(metadata, minBundleSize, startOffset, endOffset);
    this.sourceDelegate = sourceDelegate;
    this.channelFactory = channelFactory;
    boolean splittable;
    try {
      splittable = isSplittable();
    } catch (Exception e) {
      throw new RuntimeException("Failed to determine if the source is splittable", e);
    }
    checkArgument(
        splittable || startOffset == 0,
        "CompressedSources must start reading at offset 0. Requested offset: %s",
        startOffset);
  }

  /**
   * Validates that the delegate source is a valid source and that the channel factory is not null.
   */
  @Override
  public void validate() {
    super.validate();
    checkNotNull(sourceDelegate);
    sourceDelegate.validate();
    checkNotNull(channelFactory);
  }

View on GitHub (pinned to 12126d8942)

Solutions

1. Fix or wrap the delegate source's isSplittable() so it does not throw; inspect the cause via getCause().
2. Ensure the delegate's file is accessible/readable before constructing the CompressedSource.
3. For custom sources, make isSplittable() return false safely for compressed/unsplittable files instead of throwing.

Example fix

// before
@Override protected boolean isSplittable() throws Exception {
  return Files.size(path) > 0; // throws if path inaccessible
}
// after
@Override protected boolean isSplittable() throws Exception {
  try { return Files.size(path) > 0; } catch (IOException e) { return false; }
}

// verify delegate source is readable before constructing
try (FileChannel ch = FileChannel.open(path)) { /* accessible */ }

try {
  CompressedSource src = new CompressedSource(delegate, offset, factory);
} catch (RuntimeException e) {
  logger.error("splittable check failed", e.getCause()); // inspect cause
}

When it happens

Trigger: Constructing a CompressedSource whose delegate FileBasedSource.isSplittable() throws (e.g., a custom source throwing in isSplittable, or IO errors during filename/mode checks); creating sub-sources during dynamic work rebalancing.

Common situations: Custom FileBasedSource implementations with buggy isSplittable(); compressed inputs over flaky filesystems where metadata checks fail; version-mismatched delegate sources.

Understand the failure class

Background: "This is a bug, please report it": internal invariant violations, unreachable panics, and SNH errors explained — this error's family across 47 libraries.

Related errors

• Unsupported compression type: " + compression
• Must resolve compression into a concrete value before callin
• AUTO is applicable only to reading files
• Writing ZIP files is currently unsupported
• AUTO is not supported for writing

AI-assisted analysis of apache/beam@12126d8942 (2026-09-13). Data as JSON: /api/errors/8546b7b858fa2d85. Report an issue: GitHub.