{"record":{"id":"8b1a7102d59ba7cf","repo":"apache/beam","slug":"error-in-computing-splits-getsplits-returns-null","errorCode":null,"errorMessage":"Error in computing splits, getSplits() returns null.","messagePattern":"Error in computing splits, getSplits\\(\\) returns null\\.","errorType":"exception","errorClass":"IOException","httpStatus":null,"severity":"error","filePath":"sdks/java/io/hadoop-format/src/main/java/org/apache/beam/sdk/io/hadoop/format/HadoopFormatIO.java","lineNumber":795,"sourceCode":"        return boundedSourceEstimatedSize;\n      }\n      return inputSplit.getSplit().getLength();\n    }\n\n    /**\n     * This is a helper function to compute splits. This method will also calculate size of the data\n     * being read. Note: This method is executed exactly once and the splits are retrieved and\n     * cached in this. These splits are further used by split() and getEstimatedSizeBytes().\n     */\n    @VisibleForTesting\n    void computeSplitsIfNecessary() throws IOException, InterruptedException {\n      if (inputSplits != null) {\n        return;\n      }\n      createInputFormatInstance();\n      List<InputSplit> splits = inputFormatObj.getSplits(Job.getInstance(conf.get()));\n      if (splits == null) {\n        throw new IOException(\"Error in computing splits, getSplits() returns null.\");\n      }\n      if (splits.isEmpty()) {\n        throw new IOException(\"Error in computing splits, getSplits() returns a empty list\");\n      }\n      boundedSourceEstimatedSize = 0;\n      inputSplits = new ArrayList<>();\n      for (InputSplit inputSplit : splits) {\n        if (inputSplit == null) {\n          throw new IOException(\n              \"Error in computing splits, split is null in InputSplits list \"\n                  + \"populated by getSplits() : \");\n        }\n        boundedSourceEstimatedSize += inputSplit.getLength();\n        inputSplits.add(new SerializableSplit(inputSplit));\n      }\n    }\n\n    /**","sourceCodeStart":777,"sourceCodeEnd":813,"githubUrl":"https://github.com/apache/beam/blob/12126d8942aaf848030c478b4c6a28c6af861c66/sdks/java/io/hadoop-format/src/main/java/org/apache/beam/sdk/io/hadoop/format/HadoopFormatIO.java#L777-L813","documentation":"BoundedSource.computeSplitsIfNecessary() calls the Hadoop InputFormat's getSplits(); a null result means the InputFormat violated its contract, so an IOException(\"Error in computing splits, getSplits() returns null.\") is thrown. Beam requires a non-null split list to build its BoundedSource list.","triggerScenarios":"split() or getEstimatedSizeBytes() triggering computeSplitsIfNecessary when inputFormatObj.getSplits(Job) returns null — typically a custom/buggy InputFormat implementation.","commonSituations":"Custom InputFormat returning null on empty inputs instead of an empty list; version-mismatched InputFormat implementations expecting older Hadoop semantics; misconfigured input paths causing the format's own logic to fail silently and return null.","solutions":["Fix the InputFormat implementation to return an empty List<InputSplit> instead of null when there are no splits.","Verify input paths/configuration (input dir exists, format-specific configs set) so getSplits() computes normally.","Check Hadoop client version compatibility between the InputFormat and the job's Hadoop dependencies.","Catch the IOException in caller code and fall back to a non-Hadoop source if the InputFormat is known-buggy."],"exampleFix":"// before (custom InputFormat)\nif (dirs.length == 0) { return null; }\n// after\nif (dirs.length == 0) { return new ArrayList<InputSplit>(); }","handlingStrategy":"try-catch","validationCode":"InputFormat<?,?> f = inputFormatClass.getDeclaredConstructor().newInstance();\n// unit-test f.getSplits(job) against your real configuration before running the pipeline","typeGuard":"null","tryCatchPattern":"try { p.apply(HadoopFormatIO.<K,V>read().withConfiguration(conf)...); } catch (IOException e) { if (e.getMessage().contains(\"getSplits() returns null\")) { throw new IllegalStateException(\"InputFormat violated getSplits contract\", e); } throw e; }","preventionTips":["Unit test custom InputFormats to ensure getSplits never returns null","Return empty list, not null, for zero splits","Verify Hadoop dependency versions match the InputFormat's expectations"],"tags":["java","hadoop","inputformat","split-computation"],"backgroundTag":"empty-result-set","analyzedSha":"12126d8942aaf848030c478b4c6a28c6af861c66","analyzedAt":"2026-09-13T01:50:10.254Z","contentChangedAt":"2026-09-13T01:50:10.254Z","schemaVersion":2},"datasetVersion":"2026-09-20T03:17:13.778Z"}