{"record":{"id":"6280b2dbdb947933","repo":"languagetool-org/languagetool","slug":"frequency-out-of-range-0-255-normalizedfreq","errorCode":null,"errorMessage":"Frequency out of range (0-255): ${normalizedFreq} in word ${key}","messagePattern":"Frequency out of range \\(0-255\\): (.+?) in word (.+?)","errorType":"exception","errorClass":"RuntimeException","httpStatus":null,"severity":"error","filePath":"languagetool-tools/src/main/java/org/languagetool/tools/DictionaryBuilder.java","lineNumber":178,"sourceCode":"      String line;\n      int maxFreq = Collections.max(freqList.values());\n      double maxFreqLog = Math.log(maxFreq);\n      while ((line = br.readLine()) != null) {\n        Matcher m = pTaggerEntry.matcher(line);\n        if (m.matches()) {\n          int freq = 0;\n          String key = m.group(1);\n          if (freqList.containsKey(key)) {\n            freq = freqList.get(key);\n            freqValuesApplied++;\n          }\n          int normalizedFreq = freq;\n          if (freq > 0 && maxFreq > 255) {\n            double freqZeroToOne = Math.log(freq) / maxFreqLog;  // spread number better over the range\n            normalizedFreq = (int) (freqZeroToOne * (FREQ_RANGES_IN-1));  // 0 to 255\n          }\n          if (normalizedFreq < 0 || normalizedFreq > 255) {\n            throw new RuntimeException(\"Frequency out of range (0-255): \" + normalizedFreq + \" in word \" + key);\n          }\n          // Convert integers 0-255 to ranges A-Z, and write output \n          String freqChar = Character.toString((char) (FIRST_RANGE_CODE + normalizedFreq*FREQ_RANGES_OUT/FREQ_RANGES_IN));\n          //add separator only in speller dictionaries\n          if (useSeparator) { \n            bw.write(line + separator + freqChar + \"\\n\");  \n          } else {\n            bw.write(line + freqChar + \"\\n\");\n          }\n        }\n      }\n      System.out.println(freqList.size() + \" frequency values applied to \" + freqValuesApplied + \" word forms.\");\n    } catch (IOException e) {\n      throw new RuntimeException(\"Cannot read file: \" + dictFile.getAbsolutePath());\n    }\n    return tempFile;\n  }\n  ","sourceCodeStart":160,"sourceCodeEnd":196,"githubUrl":"https://github.com/languagetool-org/languagetool/blob/2e990059ce67d5e2a0f7f7ca5d31160c6709df4b/languagetool-tools/src/main/java/org/languagetool/tools/DictionaryBuilder.java#L160-L196","documentation":"Sanity guard in DictionaryBuilder.addFreqData: after log normalization a word's frequency maps outside the byte range 0-255, indicating a broken frequency value or corrupted freq list.","triggerScenarios":"Thrown at languagetool-tools/src/main/java/org/languagetool/tools/DictionaryBuilder.java:178 when the library encounters an invalid state.","commonSituations":"See trigger scenarios.","solutions":["Check the frequency list for malformed or extreme entries","Ensure the tagger list and frequency list cover consistent words","Recompute the frequency list from a clean corpus"],"exampleFix":null,"handlingStrategy":"validation","validationCode":null,"typeGuard":null,"tryCatchPattern":null,"preventionTips":[],"tags":[],"backgroundTag":null,"analyzedSha":"2e990059ce67d5e2a0f7f7ca5d31160c6709df4b","analyzedAt":"2026-09-06T09:20:17.015Z","contentChangedAt":"2026-09-06T09:20:17.015Z","schemaVersion":2},"datasetVersion":"2026-09-14T00:17:10.932Z"}