apache/hadoop · error · IOException

Expression is null

Error message

Expression is null

What it means

CompositeInputFormat.setFormat passes job.get("mapred.join.expr") to Parser.parse, which throws IOException("Expression is null") when the property is unset. The join expression is mandatory: without it there is no parse tree and the job can neither compute splits nor readers. The mapreduce-API variant reads "mapreduce.join.expr" instead, so setting the wrong API's property also lands here.

Source

Thrown at hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/java/org/apache/hadoop/mapred/join/Parser.java:479

    if (st.isEmpty()) {
      throw new IOException("Unmatched ')'");
    }
    st.pop();
    if (st.isEmpty() || !TType.IDENT.equals(st.peek().getType())) {
      throw new IOException("Identifier expected");
    }
    Node n = Node.forIdent(st.pop().getStr());
    n.parse(args, job);
    return new NodeToken(n);
  }

  /**
   * Given an expression and an optional comparator, build a tree of
   * InputFormats using the comparator to sort keys.
   */
  static Node parse(String expr, JobConf job) throws IOException {
    if (null == expr) {
      throw new IOException("Expression is null");
    }
    Class<? extends WritableComparator> cmpcl =
      job.getClass("mapred.join.keycomparator", null, WritableComparator.class);
    Lexer lex = new Lexer(expr);
    Stack<Token> st = new Stack<Token>();
    Token tok;
    while ((tok = lex.next()) != null) {
      if (TType.RPAREN.equals(tok.getType())) {
        st.push(reduce(st, job));
      } else {
        st.push(tok);
      }
    }
    if (st.size() == 1 && TType.CIF.equals(st.peek().getType())) {
      Node ret = st.pop().getNode();
      if (cmpcl != null) {
        ret.setKeyComparator(cmpcl);
      }

View on GitHub (pinned to 2add963021)

Solutions

1. Set the expression before submission: job.set("mapred.join.expr", CompositeInputFormat.compose("inner", fmt, paths)) — or CompositeInputFormat.JOIN_EXPR with the mapreduce variant
2. Set it on the same JobConf handed to the job client, before getSplits/getRecordReader run
3. Fail fast client-side: check job.get("mapred.join.expr") != null before submitting

Example fix

// before
job.setInputFormat(CompositeInputFormat.class);
// expression never set -> Expression is null at setFormat time

// after
job.setInputFormat(CompositeInputFormat.class);
job.set("mapred.join.expr",
    CompositeInputFormat.compose("inner", SequenceFileInputFormat.class, "/a", "/b"));

String expr = CompositeInputFormat.compose("inner",
    SequenceFileInputFormat.class, "/a", "/b");
job.set("mapred.join.expr", expr);
if (job.get("mapred.join.expr") == null) {
  throw new IllegalArgumentException("mapred.join.expr must be set before submission");
}

try {
  new CompositeInputFormat<Object>().setFormat(job);
} catch (IOException e) {
  if (e.getMessage() != null && e.getMessage().contains("null")) {
    throw new IllegalArgumentException(
        "mapred.join.expr not set (or set on the wrong JobConf/API)", e);
  }
  throw e;
}

When it happens

Trigger: Submitting a job with CompositeInputFormat as InputFormat but never setting mapred.join.expr; setting mapred.join.expr on a JobConf other than the one used for submission; using org.apache.hadoop.mapreduce.lib.join.CompositeInputFormat while setting only mapred.join.expr (or vice versa).

Common situations: Mixing the old (mapred) and new (mapreduce.lib.join) join APIs; the property set after the job was submitted; copy-paste jobs missing the compose/set step; property name typos.

Related errors

• Expression is null
• No more entry in " + f
• f + " already exists"
• Invalid specification for distributed-cache artifacts of typ
• Unable to parse '{}' as a URI, check the setting for mapredu

AI-assisted analysis of apache/hadoop@2add963021 (2026-08-22). Data as JSON: /api/errors/b12a5c0ff88dc567. Report an issue: GitHub.