apache/beam · error · IOException

Error in computing splits, getSplits() returns a empty list

Error message

Error in computing splits, getSplits() returns a empty list

What it means

computeSplitsIfNecessary() throws IOException("Error in computing splits, getSplits() returns a empty list") when the Hadoop InputFormat returns zero input splits. Beam cannot build a BoundedSource from an InputFormat that yields no splits, as there would be nothing to read.

Solutions

1. Verify the input path exists and contains readable data files (hdfs dfs -ls <path>).
2. Check format-specific filters/configs (e.g. mapreduce.input.fileinputformat.inputdir, file name filters) that might exclude all files.
3. If empty input is legitimate, skip the HadoopFormatIO read for that run or wrap the expand in a check on data availability.
4. Fix the InputFormat so it surfaces an actionable error or the job supplies correct configuration before Beam expansion.

Example fix

// before
conf.set("mapreduce.input.fileinputformat.inputdir", "hdfs://cluster/data/"); // empty dir
// after
conf.set("mapreduce.input.fileinputformat.inputdir", "hdfs://cluster/data/2026/09/12/");

Path in = new Path(conf.get("mapreduce.input.fileinputformat.inputdir"));
FileSystem fs = in.getFileSystem(conf);
if (!fs.exists(in) || fs.listStatus(in, p -> !p.getName().startsWith("_")).length == 0) { throw new IllegalStateException("No input data at " + in); }

null

try { p.apply(read); } catch (IOException e) { if (e.getMessage().contains("returns a empty list")) { LOG.warn("No input splits; skipping HadoopFormatIO read"); return; } throw e; }

When it happens

Trigger: split() or getEstimatedSizeBytes() triggering computeSplitsIfNecessary where inputFormatObj.getSplits(Job) returns an empty list — e.g. the configured input path is empty, doesn't exist, or all files are filtered out.

Common situations: Pointing the job at an empty or wrong input directory; input files hidden by the format's path filter (e.g. _-prefixed files); time-partitioned datasets where the requested window has no data; misconfigured table/database for formats like DBInputFormat with a query matching no rows.

Understand the failure class

Background: EmptyResultError / "no results found": when an API or scraper succeeds but returns zero rows — this error's family across 9 libraries.

Related errors

• Error in computing splits, getSplits() returns null.
• Cannot provide because does not implement the interface
• Could not parse the provided YAML stream into a non-trivial…
• Could not resolve empty field path
• ${e}

AI-assisted analysis of apache/beam@12126d8942 (2026-09-13). Data as JSON: /api/errors/4bcef24c795aba20. Report an issue: GitHub.

Appendix: source

Thrown at sdks/java/io/hadoop-format/src/main/java/org/apache/beam/sdk/io/hadoop/format/HadoopFormatIO.java:798

    }

    /**
     * This is a helper function to compute splits. This method will also calculate size of the data
     * being read. Note: This method is executed exactly once and the splits are retrieved and
     * cached in this. These splits are further used by split() and getEstimatedSizeBytes().
     */
    @VisibleForTesting
    void computeSplitsIfNecessary() throws IOException, InterruptedException {
      if (inputSplits != null) {
        return;
      }
      createInputFormatInstance();
      List<InputSplit> splits = inputFormatObj.getSplits(Job.getInstance(conf.get()));
      if (splits == null) {
        throw new IOException("Error in computing splits, getSplits() returns null.");
      }
      if (splits.isEmpty()) {
        throw new IOException("Error in computing splits, getSplits() returns a empty list");
      }
      boundedSourceEstimatedSize = 0;
      inputSplits = new ArrayList<>();
      for (InputSplit inputSplit : splits) {
        if (inputSplit == null) {
          throw new IOException(
              "Error in computing splits, split is null in InputSplits list "
                  + "populated by getSplits() : ");
        }
        boundedSourceEstimatedSize += inputSplit.getLength();
        inputSplits.add(new SerializableSplit(inputSplit));
      }
    }

    /**
     * Creates instance of InputFormat class. The InputFormat class name is specified in the Hadoop
     * configuration.
     */

View on GitHub (pinned to 12126d8942)