apache/hadoop · error · IOException

Expected ','

Error message

Expected ','

What it means

CNode.parse walks the comma-separated argument list of a composite operator; after each node argument the next token must be TType.COMMA. A missing or wrong separator between two function arguments throws IOException("Expected ','") — the canonical case is inner(tbl("a") tbl("b")), where the two tbl nodes reach CNode.parse as adjacent NodeTokens without a comma between them.

Source

Thrown at hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/java/org/apache/hadoop/mapred/join/Parser.java:440

        throw (IOException)new IOException().initCause(e);
      }
      for (int i = 0; i < capacity; ++i) {
        ret.add(kids.get(i).getRecordReader(spl.get(i), job, reporter));
      }
      return (ComposableRecordReader)ret;
    }

    /**
     * Parse a list of comma-separated nodes.
     */
    public void parse(List<Token> args, JobConf job) throws IOException {
      ListIterator<Token> i = args.listIterator();
      while (i.hasNext()) {
        Token t = i.next();
        t.getNode().setID(i.previousIndex() >> 1);
        kids.add(t.getNode());
        if (i.hasNext() && !TType.COMMA.equals(i.next().getType())) {
          throw new IOException("Expected ','");
        }
      }
    }

    public String toString() {
      StringBuilder sb = new StringBuilder();
      sb.append(ident + "(");
      for (Node n : kids) {
        sb.append(n.toString() + ",");
      }
      sb.setCharAt(sb.length() - 1, ')');
      return sb.toString();
    }
  }

  private static Token reduce(Stack<Token> st, JobConf job) throws IOException {
    LinkedList<Token> args = new LinkedList<Token>();
    while (!st.isEmpty() && !TType.LPAREN.equals(st.peek().getType())) {

View on GitHub (pinned to 2add963021)

Solutions

1. Put a comma between every pair of arguments
2. Build the expression with CompositeInputFormat.compose(op, fmt, paths...), which inserts commas itself
3. Dry-run setFormat(job) client-side to catch separator mistakes before submission

Example fix

// before
job.set("mapred.join.expr", "inner(tbl(F,/a) tbl(F,/b))");

// after
job.set("mapred.join.expr", "inner(tbl(F,/a),tbl(F,/b))");

String expr = CompositeInputFormat.compose("inner",
    SequenceFileInputFormat.class, "/a", "/b"); // inserts commas correctly
job.set("mapred.join.expr", expr);
try {
  new CompositeInputFormat<Text>().setFormat(job); // dry-run parse
} catch (IOException e) {
  throw new IllegalArgumentException("bad join expr: " + expr, e);
}

try {
  new CompositeInputFormat<Object>().setFormat(job);
} catch (IOException e) {
  throw new IllegalArgumentException(
      "arguments of the join operator must be comma-separated", e);
}

When it happens

Trigger: Omitting the comma between two tbl(...) nodes; separating arguments with whitespace, ';', or '&'; hand-edited expressions where a ',' was dropped during refactoring.

Common situations: String-concatenated expressions where the comma append was forgotten; merging two expressions into one operator's argument list by hand.

Related errors

• Expected nodetype
• Unexpected: " + type
• Parse error
• Unmatched ')'
• Identifier expected

AI-assisted analysis of apache/hadoop@2add963021 (2026-08-22). Data as JSON: /api/errors/1a061fab0ad94c22. Report an issue: GitHub.