apache/hadoop · error · IOException

Expected ','

Error message

Expected ','

What it means

Parser.CNode.parse (Parser.java:502) parses the comma-separated child list of a composite join operator like inner(...). After reading each child node it expects the next token to be a COMMA. Anything else (another node, a path, an unbalanced token) throws 'Expected \',\''.

Source

Thrown at hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/java/org/apache/hadoop/mapreduce/lib/join/Parser.java:502

      }
      for (int i = 0; i < capacity; ++i) {
        ret.add(kids.get(i).createRecordReader(spl.get(i), taskContext));
      }
      return (ComposableRecordReader)ret;
    }

    /**
     * Parse a list of comma-separated nodes.
     */
    public void parse(List<Token> args, Configuration conf) 
        throws IOException {
      ListIterator<Token> i = args.listIterator();
      while (i.hasNext()) {
        Token t = i.next();
        t.getNode().setID(i.previousIndex() >> 1);
        kids.add(t.getNode());
        if (i.hasNext() && !TType.COMMA.equals(i.next().getType())) {
          throw new IOException("Expected ','");
        }
      }
    }

    public String toString() {
      StringBuilder sb = new StringBuilder();
      sb.append(ident + "(");
      for (Node n : kids) {
        sb.append(n.toString() + ",");
      }
      sb.setCharAt(sb.length() - 1, ')');
      return sb.toString();
    }
  }

  private static Token reduce(Stack<Token> st, Configuration conf) 
      throws IOException {
    LinkedList<Token> args = new LinkedList<Token>();

View on GitHub (pinned to 2add963021)

Solutions

1. Separate every child node inside inner(...)/outer(...)/override(...) with a single comma: inner(tbl(...),tbl(...))
2. Remove trailing commas and stray tokens after the final child
3. Generate the expression with CompositeInputFormat.compose(op, infClass, paths...) which emits correct separators
4. Dry-run Parser.parse(expr, conf) in a unit test to catch syntax errors before submission

Example fix

// before
conf.set("mapreduce.join.expr",
  "outer(tbl(fmt, \"/a\") tbl(fmt, \"/b\"))");

// after
conf.set("mapreduce.join.expr",
  "outer(tbl(fmt, \"/a\"), tbl(fmt, \"/b\"))");

static String buildJoinExpr(String op, List<String> children) {
  for (String c : children) if (c == null || c.trim().isEmpty()) throw new IllegalArgumentException("null/empty join child");
  return op + "(" + String.join(",", children) + ")"; // always emits commas
}

try { Parser.parse(expr, conf); } catch (IOException e) { throw new IllegalArgumentException("Join expression syntax error (check commas between children): " + expr, e); }

When it happens

Trigger: Children of inner/outer/override not separated by commas: 'inner(tbl(fmt,"/a") tbl(fmt,"/b"))'; a trailing token after the last child: 'inner(a,b,)'; stray identifiers inside the operator call. The loop at Parser.java:496-504 consumes node, then i.next() must be COMMA whenever i.hasNext().

Common situations: Hand-writing mapreduce.join.expr and using spaces instead of commas; editing an expression and dropping a comma; building the string with a join/loop that omits separators on some boundary; whitespace/formatting changes that introduce extra tokens.

Related errors

• Expected ','
• Unmatched ')'
• Identifier expected
• Missing ')'
• Identifier expected

AI-assisted analysis of apache/hadoop@2add963021 (2026-08-22). Data as JSON: /api/errors/a316fa2b872e5d5d. Report an issue: GitHub.