{"record":{"id":"510c929cf2d1f9af","repo":"stanfordnlp/CoreNLP","slug":"unknown-token-in-line-n","errorCode":null,"errorMessage":"Unknown token in  (line )%n","messagePattern":"Unknown token in  \\(line \\)%n","errorType":"validation","errorClass":"IllegalArgumentException","httpStatus":null,"severity":"error","filePath":"src/edu/stanford/nlp/trees/treebank/ConfigParser.java","lineNumber":189,"sourceCode":"            String actualParam = tokens[0].trim();\n            String paramValue = tokens[1].trim();\n            if(paramTemplate.second != null) {\n              paramToken = paramTemplate.second.matcher(paramValue);\n              if(paramToken.matches()) {\n                paramsForDataset.setProperty(actualParam, paramValue);\n              } else {\n                System.err.printf(\"%s: Skipping illegal parameter value in %s (line %d)%n\", this.getClass().getName(), configFile,reader.getLineNumber());\n                break;\n              }\n            } else {\n              paramsForDataset.setProperty(actualParam, paramValue);\n            }\n          }\n        }\n        if (!matched) {\n          String error = this.getClass().getName() + \": Unknown token in \" + configFile + \" (line \" + reader.getLineNumber() + \")%n\";\n          System.err.printf(error);\n          throw new IllegalArgumentException(error);\n        }\n      }\n\n      if(paramsForDataset != null) datasetList.add(paramsForDataset);\n\n      reader.close();\n\n    } catch (FileNotFoundException e) {\n      System.err.printf(\"%s: Cannot open file %s%n\", this.getClass().getName(), configFile);\n    } catch (IOException e) {\n      System.err.printf(\"%s: Error reading %s (line %d)%n\", this.getClass().getName(), configFile, lineNum);\n    }\n  }\n\n  @Override\n  public String toString() {\n    final int numDatasets = datasetList.size();\n    StringBuilder sb = new StringBuilder(String.format(\"Loaded %d datasets: %n\",numDatasets));","sourceCodeStart":171,"sourceCodeEnd":207,"githubUrl":"https://github.com/stanfordnlp/CoreNLP/blob/1b7edd19c4d0d7b1f13a2591425b9b60a0b1af7a/src/edu/stanford/nlp/trees/treebank/ConfigParser.java#L171-L207","documentation":"ConfigParser.parse() reads a whitespace/section-delimited config file for dataset parsing parameters. When a line's token matches none of the known keywords, it reports 'Unknown token' with the file and line number and throws IllegalArgumentException, aborting config parsing.","triggerScenarios":"A config file contains an unrecognized keyword/token (typo, option from a different tool version, or a token meant for a section the parser does not know), encountered while parse() scans lines via main().","commonSituations":"Copy-pasting configuration examples from documentation for a different CoreNLP version; typos like 'tokenzie' or custom options; shell-quoting issues that merge tokens; editing the config by hand.","solutions":["Open the config file at the reported line number and fix or remove the unknown token","Check the spelling against valid tokens for your CoreNLP version (option names change between releases)","Remove options copied from incompatible tool versions or move them to the correct section","Quote/split arguments correctly so stray characters do not form bogus tokens"],"exampleFix":"// before (config file)\ndataset train\ntokenzie_per_line true   // typo\n// after\ndataset train\ntokenize_per_line true","handlingStrategy":"try-catch","validationCode":"// Pre-scan config lines against a whitelist of known tokens\nSet<String> known = Set.of(\"dataset\", \"tokenize_per_line\", ...);\nfor (String line : Files.readAllLines(configFile)) {\n  String tok = line.trim().split(\"\\\\s+\")[0];\n  if (!tok.isEmpty() && !known.contains(tok)) log.warn(\"Unknown config token: \" + tok);\n}","typeGuard":null,"tryCatchPattern":"try {\n  parser.parse();\n} catch (IllegalArgumentException e) {\n  System.err.println(e.getMessage()); // includes file and line number\n  // fix the config at the reported line, then retry or abort\n}","preventionTips":["Copy config examples only from the docs matching your CoreNLP version","Lint config files with a whitelist before running the pipeline","Avoid hand-editing configs in editors with autocorrect/whitespace transforms"],"tags":["java","config-parsing","illegal-argument","parser"],"backgroundTag":"invalid-config-value","analyzedSha":"1b7edd19c4d0d7b1f13a2591425b9b60a0b1af7a","analyzedAt":"2026-09-10T02:24:07.274Z","contentChangedAt":"2026-09-10T02:24:07.274Z","schemaVersion":2},"datasetVersion":"2026-09-17T15:17:12.973Z"}