apache/hadoop · error · IOException

Identifier expected

Error message

Identifier expected

What it means

Parser.reduce (Parser.java:529) pops the '(' during reduction and then requires the next stack entry to be an IDENT token — the operator name of the join expression (inner, outer, override, tbl, or a registered custom ident). 'Identifier expected' means the '(' had no identifier immediately before it, so the parenthesized group is not a valid function call.

Source

Thrown at hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/java/org/apache/hadoop/mapreduce/lib/join/Parser.java:529

        sb.append(n.toString() + ",");
      }
      sb.setCharAt(sb.length() - 1, ')');
      return sb.toString();
    }
  }

  private static Token reduce(Stack<Token> st, Configuration conf) 
      throws IOException {
    LinkedList<Token> args = new LinkedList<Token>();
    while (!st.isEmpty() && !TType.LPAREN.equals(st.peek().getType())) {
      args.addFirst(st.pop());
    }
    if (st.isEmpty()) {
      throw new IOException("Unmatched ')'");
    }
    st.pop();
    if (st.isEmpty() || !TType.IDENT.equals(st.peek().getType())) {
      throw new IOException("Identifier expected");
    }
    Node n = Node.forIdent(st.pop().getStr());
    n.parse(args, conf);
    return new NodeToken(n);
  }

  /**
   * Given an expression and an optional comparator, build a tree of
   * InputFormats using the comparator to sort keys.
   */
  static Node parse(String expr, Configuration conf) throws IOException {
    if (null == expr) {
      throw new IOException("Expression is null");
    }
    Class<? extends WritableComparator> cmpcl = conf.getClass(
      CompositeInputFormat.JOIN_COMPARATOR, null, WritableComparator.class);
    Lexer lex = new Lexer(expr);
    Stack<Token> st = new Stack<Token>();

View on GitHub (pinned to 2add963021)

Solutions

1. Remove grouping parentheses — every '(' must directly follow an identifier: ident(arg,arg)
2. Fix double parens after operators: inner((tbl(...))) → inner(tbl(...))
3. Generate expressions via CompositeInputFormat.compose(...) instead of hand-writing
4. Validate with Parser.parse(expr, conf) in a test before job submission

Example fix

// before
String expr = "inner((tbl(fmt, \"/a\")), tbl(fmt, \"/b\"))";

// after
String expr = "inner(tbl(fmt, \"/a\"), tbl(fmt, \"/b\"))";

static void requireIdentBeforeParen(String expr) {
  // no '(' may be preceded by '(' or ',' or start of string
  java.util.regex.Matcher m = java.util.regex.Pattern.compile("(^|[(,])\\s*\\(").matcher(expr);
  if (m.find()) throw new IllegalArgumentException("'(' must directly follow an identifier (no grouping parens)");
}

try { Parser.parse(expr, conf); } catch (IOException e) { throw new IllegalArgumentException("Join grammar error (parens are call syntax, not grouping): " + expr, e); }

When it happens

Trigger: Expressions like '(tbl(fmt,"/a"))' (paren before ident), 'inner((tbl(...)))' (double paren after operator), ',(...)' or stray '(' anywhere a node was expected: 'inner(,(tbl(...)))', 'tbl((fmt),"/a")'. The grammar treats every parenthesized group as ident(args).

Common situations: Hand-writing expressions and adding grouping parens out of habit from general math syntax (the join grammar has no grouping — parens are always call syntax); typos introducing double '((' after an operator; editing tokens around an existing paren.

Related errors

• Expected ','
• Unmatched ')'
• Identifier expected
• Missing ')'
• Expected ','

AI-assisted analysis of apache/hadoop@2add963021 (2026-08-22). Data as JSON: /api/errors/da60b122594a176e. Report an issue: GitHub.