languagetool-org/languagetool · error · IOException

A separator character (fsa.dict.separator) must be defined i

Error message

A separator character (fsa.dict.separator) must be defined in the dictionary info file.

What it means

Configuration guard in DictionaryBuilder.addFreqData: the dictionary .info file has no fsa.dict.separator property, so frequency/lemma fields cannot be separated from the encoded entry.

Source

Thrown at languagetool-tools/src/main/java/org/languagetool/tools/DictionaryBuilder.java:151

      String line;
      while ((line = br.readLine()) != null) {
        Matcher m = pFreqEntry.matcher(line);
        if (m.matches()) {
          freqList.put(m.group(3), Integer.parseInt(m.group(1)));
        }
      }
    } catch (IOException e) {
      throw new RuntimeException("Cannot read file: " + freqListFile.getAbsolutePath());
    }
  }
  
  protected File addFreqData(File dictFile, boolean useSeparator) throws IOException {
    if (!isOptionTrue("fsa.dict.frequency-included")) {
      throw new IOException("In order to use frequency data add the line 'fsa.dict.frequency-included=true' to the dictionary info file.");
    }
    String separator = getOption("fsa.dict.separator");
    if (separator == null || separator.trim().isEmpty()) {
      throw new IOException("A separator character (fsa.dict.separator) must be defined in the dictionary info file.");
    }
    File tempFile = File.createTempFile(DictionaryBuilder.class.getSimpleName(), "WithFrequencies.txt");
    tempFile.deleteOnExit();
    String encoding = getOption("fsa.dict.encoding");
    int freqValuesApplied = 0;

    try (BufferedWriter bw = new BufferedWriter(new OutputStreamWriter(new FileOutputStream(tempFile.getAbsoluteFile()), encoding));
         BufferedReader br = new BufferedReader(new InputStreamReader(new FileInputStream(dictFile.getAbsoluteFile()), encoding))) {
      String line;
      int maxFreq = Collections.max(freqList.values());
      double maxFreqLog = Math.log(maxFreq);
      while ((line = br.readLine()) != null) {
        Matcher m = pTaggerEntry.matcher(line);
        if (m.matches()) {
          int freq = 0;
          String key = m.group(1);
          if (freqList.containsKey(key)) {
            freq = freqList.get(key);

View on GitHub (pinned to 2e990059ce)

Solutions

1. Add fsa.dict.separator=<char> to the dictionary .info file
2. Use a separator character that does not occur in encoded entries

When it happens

Trigger: Thrown at languagetool-tools/src/main/java/org/languagetool/tools/DictionaryBuilder.java:151 when the library encounters an invalid state.

Common situations: See trigger scenarios.

AI-assisted analysis of languagetool-org/languagetool@2e990059ce (2026-09-06). Data as JSON: /api/errors/4e2bb5633c644504. Report an issue: GitHub.