apache/beam · error · IOException

Error in computing splits, split is null in InputSplits…

Error message

Error in computing splits, split is null in InputSplits list populated by getSplits() :

What it means

HadoopFormatIO's computeSplitsIfNecessary() calls the wrapped Hadoop InputFormat's getSplits() and validates every returned InputSplit. If any element of the list is null, it throws this IOException because a null split cannot be converted to a SerializableSplit or contribute a length to the estimated size.

Solutions

Inspect the InputFormat.getSplits() implementation and ensure it never inserts null elements into the returned list
Verify the configured InputFormat class matches the actual data source (TableInputFormat, TextInputFormat, etc.)
Check that job configuration (paths, table name, filters) is valid so the InputFormat can produce complete splits
If data is legitimately empty, return an empty list instead of a list containing nulls

Example fix

// before
List<InputSplit> splits = new ArrayList<>(numSplits);
// after
List<InputSplit> splits = new ArrayList<>();
for (...) { splits.add(computeSplit(i)); } // never add(null)

Defensive patterns

Strategy: validation

Validate before calling

List<InputSplit> splits = inputFormat.getSplits(jobConf);
if (splits == null || splits.stream().anyMatch(Objects::isNull)) {
  throw new IllegalArgumentException("InputFormat returned empty/null splits; check job config and data");
}

Type guard

static boolean hasNullSplits(List<InputSplit> splits) {
  return splits == null || splits.stream().anyMatch(Objects::isNull);
}

Try / catch

try {
  source.split(desiredSize, options);
} catch (IOException e) {
  if (e.getMessage().contains("split is null")) {
    throw new IllegalStateException("Bad InputFormat: getSplits() produced null entries", e);
  }
  throw e;
}

Prevention

Unit-test custom InputFormat.getSplits() to assert no null entries
Return an empty list for genuinely empty data, never nulls
Log the split list size and contents before handing it to Beam

When it happens

Trigger: A user-implemented or buggy InputFormat.getSplits() returns a List<InputSplit> containing null entries; called automatically when split(), getEstimatedSizeBytes(), or a test runner enumerates the BoundedSource for a HadoopFormatIO.read() transform.

Common situations: Custom InputFormat implementations that pre-size their splits list (new ArrayList<>(n)) without filling all slots; Hadoop InputFormats that return placeholder nulls when no data matches the filter; wrong InputFormat class configured so its getSplits() is partially initialized.

Related errors

AI-assisted analysis of apache/beam@12126d8942 (2026-09-13). Data as JSON: /api/errors/8ba4bb4552507a8f. Report an issue: GitHub.

Appendix: source

Thrown at sdks/java/io/hadoop-format/src/main/java/org/apache/beam/sdk/io/hadoop/format/HadoopFormatIO.java:804

     */
    @VisibleForTesting
    void computeSplitsIfNecessary() throws IOException, InterruptedException {
      if (inputSplits != null) {
        return;
      }
      createInputFormatInstance();
      List<InputSplit> splits = inputFormatObj.getSplits(Job.getInstance(conf.get()));
      if (splits == null) {
        throw new IOException("Error in computing splits, getSplits() returns null.");
      }
      if (splits.isEmpty()) {
        throw new IOException("Error in computing splits, getSplits() returns a empty list");
      }
      boundedSourceEstimatedSize = 0;
      inputSplits = new ArrayList<>();
      for (InputSplit inputSplit : splits) {
        if (inputSplit == null) {
          throw new IOException(
              "Error in computing splits, split is null in InputSplits list "
                  + "populated by getSplits() : ");
        }
        boundedSourceEstimatedSize += inputSplit.getLength();
        inputSplits.add(new SerializableSplit(inputSplit));
      }
    }

    /**
     * Creates instance of InputFormat class. The InputFormat class name is specified in the Hadoop
     * configuration.
     */
    @SuppressWarnings("WeakerAccess")
    protected void createInputFormatInstance() throws IOException {
      if (inputFormatObj == null) {
        try {
          taskAttemptContext = new TaskAttemptContextImpl(conf.get(), new TaskAttemptID());
          inputFormatObj =

View on GitHub (pinned to 12126d8942)