{"record":{"id":"2b6f5ab56f5935c7","repo":"apache/hadoop","slug":"error-gathering-splits-from-child-rreader","errorCode":null,"errorMessage":"Error gathering splits from child RReader","messagePattern":"Error gathering splits from child RReader","errorType":"exception","errorClass":"IOException","httpStatus":null,"severity":"error","filePath":"hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/java/org/apache/hadoop/mapred/join/Parser.java","lineNumber":382,"sourceCode":"\n    public void setKeyComparator(Class<? extends WritableComparator> cmpcl) {\n      super.setKeyComparator(cmpcl);\n      for (Node n : kids) {\n        n.setKeyComparator(cmpcl);\n      }\n    }\n\n    /**\n     * Combine InputSplits from child InputFormats into a\n     * {@link CompositeInputSplit}.\n     */\n    public InputSplit[] getSplits(JobConf job, int numSplits)\n        throws IOException {\n      InputSplit[][] splits = new InputSplit[kids.size()][];\n      for (int i = 0; i < kids.size(); ++i) {\n        final InputSplit[] tmp = kids.get(i).getSplits(job, numSplits);\n        if (null == tmp) {\n          throw new IOException(\"Error gathering splits from child RReader\");\n        }\n        if (i > 0 && splits[i-1].length != tmp.length) {\n          throw new IOException(\"Inconsistent split cardinality from child \" +\n              i + \" (\" + splits[i-1].length + \"/\" + tmp.length + \")\");\n        }\n        splits[i] = tmp;\n      }\n      final int size = splits[0].length;\n      CompositeInputSplit[] ret = new CompositeInputSplit[size];\n      for (int i = 0; i < size; ++i) {\n        ret[i] = new CompositeInputSplit(splits.length);\n        for (int j = 0; j < splits.length; ++j) {\n          ret[i].add(splits[j][i]);\n        }\n      }\n      return ret;\n    }\n","sourceCodeStart":364,"sourceCodeEnd":400,"githubUrl":"https://github.com/apache/hadoop/blob/2add9630210752f88ceb1bb74eb65e37bf41da8e/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/java/org/apache/hadoop/mapred/join/Parser.java#L364-L400","documentation":"Parser.CNode.getSplits asks each child node for its splits and throws IOException(\"Error gathering splits from child RReader\") when a child returns null instead of an array. The InputFormat.getSplits contract says implementations return an array (possibly empty), never null, so in practice this means a custom or third-party InputFormat inside a tbl(...) node violates that contract. The message text is misleading — the null comes from the child InputFormat, not a RecordReader.","triggerScenarios":"A custom InputFormat whose getSplits returns null on edge cases (empty input, swallowed error); a delegating wrapper InputFormat that forwards a null from its inner format without checking.","commonSituations":"Custom input formats bolted into a join expression; formats written for a client that tolerated null splits; formats that return null for empty directories instead of an empty array.","solutions":["Fix the child InputFormat to return an empty array instead of null","Wrap the offending format in a delegating InputFormat that null-guards getSplits","Unit-test the custom format standalone with 0-, 1-, and N-file inputs before using it in a join"],"exampleFix":"// before (custom InputFormat)\npublic InputSplit[] getSplits(JobConf job, int numSplits) throws IOException {\n  List<InputSplit> l = computeSplits(job); // may be null\n  return l == null ? null : l.toArray(new InputSplit[0]);\n}\n\n// after\npublic InputSplit[] getSplits(JobConf job, int numSplits) throws IOException {\n  List<InputSplit> l = computeSplits(job);\n  return (l == null) ? new InputSplit[0] : l.toArray(new InputSplit[0]);\n}","handlingStrategy":"validation","validationCode":"// preflight: every child InputFormat must return a non-null split array\nfor (Path p : joinSources) {\n  InputFormat<?, ?> child = getChildFormat(p);\n  InputSplit[] s = child.getSplits(job, 1);\n  if (s == null) {\n    throw new IOException(\"child InputFormat \" + child.getClass()\n        + \" returned null from getSplits\");\n  }\n}","typeGuard":null,"tryCatchPattern":"try {\n  return cnodeOrFormat.getSplits(job, numSplits);\n} catch (IOException e) {\n  if (e.getMessage() != null && e.getMessage().contains(\"gathering splits\")) {\n    throw new IOException(\"a child InputFormat returned null splits; fix its getSplits()\", e);\n  }\n  throw e;\n}","preventionTips":["Custom InputFormats must return an array, empty if needed, never null","Unit-test custom formats with empty inputs before joining them","Wrap third-party formats in a null-guarding delegator"],"tags":["hadoop","mapreduce","join","inputformat","null-return"],"backgroundTag":"null-return-value","analyzedSha":"2add9630210752f88ceb1bb74eb65e37bf41da8e","analyzedAt":"2026-08-22T19:55:07.957Z","schemaVersion":2},"datasetVersion":"2026-08-23T01:17:44.959Z"}