No multi-word rules provided. No inferMultiWordRules flag…

Error message

No multi-word rules provided. No inferMultiWordRules flag validated. Not inferring rules from training.

What it means

When StatTokSentTrainer has no multiWordRulesFile and the inferMultiWordRules flag is not enabled, it cannot obtain multi-word tokenization rules and logs this warning, continuing with an empty rule map. Training proceeds but multi-word expressions (e.g. 'del', 'au') will not be handled, degrading tokenizer quality on languages that need them.

Solutions

Pass -multiWordRulesFile pointing to rules generated by BuildMultiWordRules.
Or enable the inferMultiWordRules flag so rules are inferred from the training set automatically.
Generate rules first with BuildMultiWordRules if you don't have a rules file.
Accept the warning only if the target language truly has no multi-word tokens.

Example fix

// before
java -cp ... StatTokSentTrainer -trainFile train.conllu -serializeTo model.gz
// after
java -cp ... StatTokSentTrainer -trainFile train.conllu -multiWordRulesFile mw-rules.txt -serializeTo model.gz

Defensive patterns

Strategy: validation

Validate before calling

boolean hasRules = argsContains(args, "multiWordRulesFile");
boolean infer = Boolean.parseBoolean(props.getProperty("inferMultiWordRules", "false"));
if (!hasRules && !infer) {
    System.err.println("WARN: no multi-word rules source; pass -multiWordRulesFile or enable inferMultiWordRules");
}

Prevention

For languages with contractions (fr, es, pt, gl), always supply multi-word rules.
Generate rules with BuildMultiWordRules as a pipeline step before training.
Treat this warning as fatal in CI for rule-sensitive languages.

When it happens

Trigger: Running StatTokSentTrainer with neither -multiWordRulesFile nor the inferMultiWordRules option set true, hitting the else-branch warn at line 439.

Common situations: Training French/Spanish/Galician tokenizers where multi-word rules matter but the flag was forgotten; users who read BuildMultiWordRules output but forgot to pass the resulting file; misreading the flag name and passing an unrecognized variant.

Understand the failure class

Background: "--flag is required" and "must specify" CLI errors: how missing-required-flag validation works and how to fix it — this error's family across 20 libraries.

Related errors

AI-assisted analysis of stanfordnlp/CoreNLP@1b7edd19c4 (2026-09-10). Data as JSON: /api/errors/18b93a241cfb9747. Report an issue: GitHub.

Appendix: source

Thrown at src/edu/stanford/nlp/process/stattok/StatTokSentTrainer.java:439

      logger.err("Error: No training file provided in properties or via command line.");
      return;
    }

    Map<String, String[]> multiWordRules = new HashMap<String, String[]>();
    // Read or generate multi-word rules
    if (multiWordRulesFile != null){
      if (inferMultiWordRules){
        logger.warn("Conflicting properties. Multi-word rules file will be considered.");
      }
      logger.info("Reading Multi-Word rules file ... ");
      multiWordRules = trainTokenizer.readMultiWordRules(multiWordRulesFile);
    } else {
      if (inferMultiWordRules){
        logger.info("Inferring Multi-Word rules from training set ... ");
        multiWordRules = trainTokenizer.inferMultiWordRules(trainFile);
      }
      else{
        logger.warn("No multi-word rules provided. No inferMultiWordRules flag validated. Not inferring rules from training.");
      }
    }

    // Generate training file from ConLL-U
    logger.info("Creating training set from "+trainFile);
    ArrayList<Pair<String, String>> classCharText = trainTokenizer.fileToTrainSet(trainFile,multiWordRules);
    logger.info("Adding Features");
    List<String> trainingInput = trainTokenizer.addFeatures(classCharText, windowSize);
    logger.info("Training is ready.");
		
    // Write temporary training data on file
    File trainFileIOB = File.createTempFile("training.", ".iob");
    trainFileIOB.deleteOnExit();
    OutputStreamWriter fileWriter = new OutputStreamWriter(new FileOutputStream(trainFileIOB), StandardCharsets.UTF_8);
    for (String line : trainingInput){
      fileWriter.write(line+System.lineSeparator());
    }
    fileWriter.close();

View on GitHub (pinned to 1b7edd19c4)