apache/hadoop · error · IOException

Expected nodetype

Error message

Expected nodetype

What it means

Parser tokens are a tagged union; only NodeToken (TType.CIF) carries a Node, and Token.getNode() on any other token throws IOException("Expected nodetype"). In practice this fires inside CNode.parse (Parser.java:437) while interpreting the argument list of a composite operator (inner/outer/override or a mapred.join.define identifier): every argument must itself be a function node such as tbl(...), and a bare identifier, number, or quoted string in that position triggers the error.

Source

Thrown at hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/java/org/apache/hadoop/mapred/join/Parser.java:88

  public enum TType { CIF, IDENT, COMMA, LPAREN, RPAREN, QUOT, NUM, }

  /**
   * Tagged-union type for tokens from the join expression.
   * @see Parser.TType
   */
  @InterfaceAudience.Public
  @InterfaceStability.Evolving
  public static class Token {

    private TType type;

    Token(TType type) {
      this.type = type;
    }

    public TType getType() { return type; }
    public Node getNode() throws IOException {
      throw new IOException("Expected nodetype");
    }
    public double getNum() throws IOException {
      throw new IOException("Expected numtype");
    }
    public String getStr() throws IOException {
      throw new IOException("Expected strtype");
    }
  }

  @InterfaceAudience.Public
  @InterfaceStability.Evolving
  public static class NumToken extends Token {
    private double num;
    public NumToken(double num) {
      super(TType.NUM);
      this.num = num;
    }
    public double getNum() { return num; }

View on GitHub (pinned to 2add963021)

Solutions

1. Wrap every argument in a function node, normally tbl(<InputFormat class>,\"<path>\")
2. Build the expression programmatically with CompositeInputFormat.compose(op, InputFormat.class, paths...), which always emits well-formed arguments
3. Dry-run the expression client-side via new CompositeInputFormat().setFormat(job) to fail before job submission

Example fix

// before
job.set("mapred.join.expr", "inner(left, right)");

// after
job.set("mapred.join.expr",
    CompositeInputFormat.compose("inner", SequenceFileInputFormat.class,
        "/join/left", "/join/right"));

String expr = "inner(tbl(org.apache.hadoop.mapred.SequenceFileInputFormat,\"/a\"),"
    + "tbl(org.apache.hadoop.mapred.SequenceFileInputFormat,\"/b\"))";
job.set("mapred.join.expr", expr);
try {
  new CompositeInputFormat<Text>().setFormat(job); // dry-run parse client-side
} catch (IOException e) {
  throw new IllegalArgumentException("bad join expr: " + expr, e);
}

boolean isNodeToken(Parser.Token t) {
  return Parser.TType.CIF.equals(t.getType()); // only NodeToken carries a Node
}

try {
  new CompositeInputFormat<Object>().setFormat(job);
} catch (IOException e) {
  throw new IllegalArgumentException(
      "mapred.join.expr must contain only function-call arguments like tbl(...)", e);
}

When it happens

Trigger: Expressions like inner(left, right) where the arguments are bare IDENT tokens instead of tbl(...) calls; outer(tbl("a"), "b") with a quoted string argument; expressions built by string concatenation that drop the tbl(...) wrapper around a source.

Common situations: Hand-writing mapred.join.expr instead of using CompositeInputFormat.compose(); copying an example expression and deleting a tbl(...) wrapper while refactoring.

Related errors

• Unexpected: " + type
• Parse error
• Expected ','
• Expected numtype
• Expected strtype

AI-assisted analysis of apache/hadoop@2add963021 (2026-08-22). Data as JSON: /api/errors/49b1ad6cc275e647. Report an issue: GitHub.