stanfordnlp/CoreNLP · error · RuntimeIOException

TokensRegexNERAnnotator

Error message

TokensRegexNERAnnotator ${name}: Error opening the common words file: ${commonWordsFile}

What it means

TokensRegexNERAnnotator reads the 'commonWords' option file (words ignored during NER matching). Opening it via IOUtils.readerFromString throws IOException on missing/unreadable files, which is wrapped in RuntimeIOException naming the annotator name and file path.

Solutions

1. Verify the file exists at the exact path (use absolute paths during debugging).
2. If loading from classpath, use a path that IOUtils can resolve, e.g. prefix with the proper resource location, and confirm the resource is in the jar.
3. Check file read permissions for the process user.
4. Remove the commonWords option if you do not need a common-words suppression list.

Example fix

// before
props.setProperty("tokensregexner.rules.commonWords", "data/common_words.txt");
// after (verify file exists, or use classpath resource)
props.setProperty("tokensregexner.rules.commonWords", "/etc/corenlp/common_words.txt");

String cw = props.getProperty("tokensregexner.rules.commonWords");
if (cw != null && !cw.isEmpty() && !new File(cw).canRead() && getClass().getResource(cw) == null)
  throw new IllegalStateException("commonWords file not readable: " + cw);

boolean readableResource(String path) { return new File(path).canRead() || Thread.currentThread().getContextClassLoader().getResource(path) != null; }

try { new TokensRegexNERAnnotator(props); } catch (RuntimeIOException e) { if (e.getMessage().contains("Error opening the common words file")) { log.error("Missing commonWords file: " + e.getMessage()); } throw e; }

When it happens

Trigger: Setting tokensregexner.<name>.commonWords (or the default prefix) to a file path or URL that does not exist, cannot be opened, or is not readable at annotation-construction time.

Common situations: Relative path resolved from a different working directory; resource file not packaged in the jar; typo in the filename; running in a container where the data files were not mounted.

Understand the failure class

Background: "open() failed", "failed to open file", "cannot create file" — what a file open error means and how to fix it — this error's family across 42 libraries.

Related errors

• Error reading saved links
• Error creating data exporter
• Error setting up training
• RuntimeException wrapping IOException
• RuntimeIOException wrapping IOException

AI-assisted analysis of stanfordnlp/CoreNLP@1b7edd19c4 (2026-09-10). Data as JSON: /api/errors/0164f2ac2e26090b. Report an issue: GitHub.

Appendix: source

Thrown at src/edu/stanford/nlp/pipeline/TokensRegexNERAnnotator.java:252

  public TokensRegexNERAnnotator(String name, Properties properties) {
    String prefix = ! StringUtils.isNullOrEmpty(name) ? name + '.': "";
    String backgroundSymbol = properties.getProperty(prefix + "backgroundSymbol", DEFAULT_BACKGROUND_SYMBOL);
    String[] backgroundSymbols = COMMA_DELIMITERS_PATTERN.split(backgroundSymbol);
    String mappingFiles = properties.getProperty(prefix + "mapping", DefaultPaths.DEFAULT_KBP_TOKENSREGEX_NER_SETTINGS);
    String[] mappings = processListMappingFiles(mappingFiles);
    String validPosRegex = properties.getProperty(prefix + "validpospattern");
    this.posMatchType = PosMatchType.valueOf(properties.getProperty(prefix + "posmatchtype",
            DEFAULT_POS_MATCH_TYPE.name()));
    String commonWordsFile = properties.getProperty(prefix + "commonWords");
    commonWords = new HashSet<>();
    if (commonWordsFile != null) {
      try (BufferedReader reader = IOUtils.readerFromString(commonWordsFile)) {
        for (String line; (line = reader.readLine()) != null; ) {
          commonWords.add(line);
        }
      } catch (IOException ex) {
        throw new RuntimeIOException("TokensRegexNERAnnotator " + name
            + ": Error opening the common words file: " + commonWordsFile, ex);
      }
    }

    String headerProp = properties.getProperty(prefix + "mapping.header", defaultHeader);
    boolean readHeaderFromFile = headerProp.equalsIgnoreCase("true");
    String[] annotationFieldnames = null;
    String[] headerFields = null;
    if (readHeaderFromFile) {
      annotationFieldnames = StringUtils.EMPTY_STRING_ARRAY;
      annotationFields = new ArrayList<>();
      // Set the read header property of each file to true
      for (int i = 0; i < mappings.length; i++) {
        String mappingLine = mappings[i];
        if ( ! mappingLine.contains("header")) {
          mappingLine = "header=true, " + mappingLine;
          mappings[i] = mappingLine;
        } else if ( ! Pattern.compile("header\\s*=\\s*true").matcher(mappingLine.toLowerCase()).find()) {

View on GitHub (pinned to 1b7edd19c4)