apache/hadoop · error · IOException

Expression is null

Error message

Expression is null

What it means

CompositeInputFormat.setFormat reads the join expression from the 'mapreduce.join.expr' property and hands it to Parser.parse (Parser.java:545-547). If the property is unset (conf.get returns null), the parser immediately throws 'Expression is null'. It means the job never defined its join expression.

Source

Thrown at hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/java/org/apache/hadoop/mapreduce/lib/join/Parser.java:542

    if (st.isEmpty()) {
      throw new IOException("Unmatched ')'");
    }
    st.pop();
    if (st.isEmpty() || !TType.IDENT.equals(st.peek().getType())) {
      throw new IOException("Identifier expected");
    }
    Node n = Node.forIdent(st.pop().getStr());
    n.parse(args, conf);
    return new NodeToken(n);
  }

  /**
   * Given an expression and an optional comparator, build a tree of
   * InputFormats using the comparator to sort keys.
   */
  static Node parse(String expr, Configuration conf) throws IOException {
    if (null == expr) {
      throw new IOException("Expression is null");
    }
    Class<? extends WritableComparator> cmpcl = conf.getClass(
      CompositeInputFormat.JOIN_COMPARATOR, null, WritableComparator.class);
    Lexer lex = new Lexer(expr);
    Stack<Token> st = new Stack<Token>();
    Token tok;
    while ((tok = lex.next()) != null) {
      if (TType.RPAREN.equals(tok.getType())) {
        st.push(reduce(st, conf));
      } else {
        st.push(tok);
      }
    }
    if (st.size() == 1 && TType.CIF.equals(st.peek().getType())) {
      Node ret = st.pop().getNode();
      if (cmpcl != null) {
        ret.setKeyComparator(cmpcl);
      }

View on GitHub (pinned to 2add963021)

Solutions

1. Set the expression before submission: job.getConfiguration().set("mapreduce.join.expr", CompositeInputFormat.compose("inner", TextInputFormat.class, "/a", "/b"))
2. Or call new CompositeInputFormat<>().setFormat(conf) after the property is set (it also registers default identifiers)
3. Verify you use the new-API property name mapreduce.join.expr (old API: mapred.join.expr in org.apache.hadoop.mapred.join)
4. Print/log conf.get("mapreduce.join.expr") right before job.submit() to confirm it is non-null on the submitted config

Example fix

// before
job.setInputFormatClass(CompositeInputFormat.class);
// forgot the expression -> 'Expression is null' at getSplits/createRecordReader time

// after
job.setInputFormatClass(CompositeInputFormat.class);
job.getConfiguration().set("mapreduce.join.expr",
  CompositeInputFormat.compose("inner", TextInputFormat.class, "/join/a", "/join/b"));

static Configuration withJoinExpr(Configuration conf, String op, Class<? extends InputFormat<?>> inf, String... paths) {
  if (conf.get("mapreduce.join.expr") == null)
    conf.set("mapreduce.join.expr", CompositeInputFormat.compose(op, inf, paths));
  return conf;
}

try { cif.getSplits(job); } catch (IOException e) { if ("Expression is null".equals(e.getMessage())) throw new IllegalStateException("mapreduce.join.expr not set — configure the join expression in the driver", e); throw e; }

When it happens

Trigger: Using CompositeInputFormat as the job's InputFormat (job.setInputFormatClass or the mapred equivalent) without calling CompositeInputFormat.setFormat or setting 'mapreduce.join.expr'; setting the property on a different Configuration instance than the one the job uses; relying on mapred.join.expr (old API name) with the new API parser.

Common situations: New join jobs missing the expression setup line; refactors that drop the conf.set("mapreduce.join.expr", ...) call; copying examples that use the old mapred API (org.apache.hadoop.mapred.join.CompositeInputFormat uses 'mapred.join.expr') into new-API code; configuration set on the job's Configuration after Job submission or on a template conf that is not propagated.

Related errors

• Expression is null
• No more entry in " + f
• f + " already exists"
• Invalid specification for distributed-cache artifacts of typ
• Unable to parse '{}' as a URI, check the setting for mapredu

AI-assisted analysis of apache/hadoop@2add963021 (2026-08-22). Data as JSON: /api/errors/da575bd6ae12c605. Report an issue: GitHub.