apache/beam · error · IOException

Error in computing splits, getSplits() returns null.

Error message

Error in computing splits, getSplits() returns null.

What it means

BoundedSource.computeSplitsIfNecessary() calls the Hadoop InputFormat's getSplits(); a null result means the InputFormat violated its contract, so an IOException("Error in computing splits, getSplits() returns null.") is thrown. Beam requires a non-null split list to build its BoundedSource list.

Solutions

Fix the InputFormat implementation to return an empty List<InputSplit> instead of null when there are no splits.
Verify input paths/configuration (input dir exists, format-specific configs set) so getSplits() computes normally.
Check Hadoop client version compatibility between the InputFormat and the job's Hadoop dependencies.
Catch the IOException in caller code and fall back to a non-Hadoop source if the InputFormat is known-buggy.

Example fix

// before (custom InputFormat)
if (dirs.length == 0) { return null; }
// after
if (dirs.length == 0) { return new ArrayList<InputSplit>(); }

Defensive patterns

Strategy: try-catch

Validate before calling

InputFormat<?,?> f = inputFormatClass.getDeclaredConstructor().newInstance();
// unit-test f.getSplits(job) against your real configuration before running the pipeline

Type guard

null

Try / catch

try { p.apply(HadoopFormatIO.<K,V>read().withConfiguration(conf)...); } catch (IOException e) { if (e.getMessage().contains("getSplits() returns null")) { throw new IllegalStateException("InputFormat violated getSplits contract", e); } throw e; }

Prevention

Unit test custom InputFormats to ensure getSplits never returns null
Return empty list, not null, for zero splits
Verify Hadoop dependency versions match the InputFormat's expectations

When it happens

Trigger: split() or getEstimatedSizeBytes() triggering computeSplitsIfNecessary when inputFormatObj.getSplits(Job) returns null — typically a custom/buggy InputFormat implementation.

Common situations: Custom InputFormat returning null on empty inputs instead of an empty list; version-mismatched InputFormat implementations expecting older Hadoop semantics; misconfigured input paths causing the format's own logic to fail silently and return null.

Understand the failure class

Background: EmptyResultError / "no results found": when an API or scraper succeeds but returns zero rows — this error's family across 9 libraries.

Related errors

AI-assisted analysis of apache/beam@12126d8942 (2026-09-13). Data as JSON: /api/errors/8b1a7102d59ba7cf. Report an issue: GitHub.

Appendix: source

Thrown at sdks/java/io/hadoop-format/src/main/java/org/apache/beam/sdk/io/hadoop/format/HadoopFormatIO.java:795

        return boundedSourceEstimatedSize;
      }
      return inputSplit.getSplit().getLength();
    }

    /**
     * This is a helper function to compute splits. This method will also calculate size of the data
     * being read. Note: This method is executed exactly once and the splits are retrieved and
     * cached in this. These splits are further used by split() and getEstimatedSizeBytes().
     */
    @VisibleForTesting
    void computeSplitsIfNecessary() throws IOException, InterruptedException {
      if (inputSplits != null) {
        return;
      }
      createInputFormatInstance();
      List<InputSplit> splits = inputFormatObj.getSplits(Job.getInstance(conf.get()));
      if (splits == null) {
        throw new IOException("Error in computing splits, getSplits() returns null.");
      }
      if (splits.isEmpty()) {
        throw new IOException("Error in computing splits, getSplits() returns a empty list");
      }
      boundedSourceEstimatedSize = 0;
      inputSplits = new ArrayList<>();
      for (InputSplit inputSplit : splits) {
        if (inputSplit == null) {
          throw new IOException(
              "Error in computing splits, split is null in InputSplits list "
                  + "populated by getSplits() : ");
        }
        boundedSourceEstimatedSize += inputSplit.getLength();
        inputSplits.add(new SerializableSplit(inputSplit));
      }
    }

    /**

View on GitHub (pinned to 12126d8942)