languagetool-org/languagetool · error · UnsupportedOperationException

Too many values (" + valueCount + "), the storage needs adju

Error message

Too many values (" + valueCount + "), the storage needs adjusting

What it means

Capacity guard in the ManualSynthesizer constructor: after loading the lemma/POS/word-form mapping from the input stream, the total number of stored values exceeds the maximum offset the compact offset encoding can represent (MAX_OFFSET). The input at fault is the synthesizer data file being loaded; the implementation's storage scheme, not the caller, is the limit.

Source

Thrown at languagetool-core/src/main/java/org/languagetool/synthesis/ManualSynthesizer.java:77

  private static final int MAX_OFFSET = (1 << 32 - OFFSET_SHIFT) - 1;
  private static final int ENTRY_SIZE = 3;
  private final String[] data;

  /** A map from lemma+POS hashes to encoded lemma+POS+word tuple offsets in {@link #data} */
  private final Int2IntMap map;

  private final static String DEFAULT_SEPARATOR = "\t";

  public ManualSynthesizer(InputStream inputStream) throws IOException {
    Map<TaggedWord, List<String>> mapping = loadMapping(inputStream);
    Int2ObjectOpenHashMap<List<Triple<String, String, String>>> byHash = groupByHash(mapping);

    map = new Int2IntOpenHashMap(byHash.size());
    int valueCount = mapping.values().stream().mapToInt(v -> v.size()).sum();
    int firstIndex = ENTRY_SIZE; // skip an entry, as 0 means an absent value in TObjectIntHashMap
    data = new String[valueCount * ENTRY_SIZE + firstIndex];
    if (valueCount > MAX_OFFSET) {
      throw new UnsupportedOperationException("Too many values (" + valueCount + "), the storage needs adjusting");
    }
    byHash.int2ObjectEntrySet().fastForEach(new Consumer<Int2ObjectMap.Entry<List<Triple<String, String, String>>>>() {
      int index = firstIndex;

      @Override
      public void accept(Int2ObjectMap.Entry<List<Triple<String, String, String>>> listEntry) {
        int hash = listEntry.getIntKey();
        List<Triple<String, String, String>> value = listEntry.getValue();
        if (value.size() > MAX_LENGTH) {
          throw new UnsupportedOperationException(
            "Too many lemmas (" + value.size() + " for the same hash " + value + ", the storage needs adjusting");
        }
        map.put(hash, ((index / ENTRY_SIZE) << OFFSET_SHIFT) | value.size());
        for (Triple<String, String, String> triple : value) {
          data[index++] = intern(triple.getLeft());
          data[index++] = intern(triple.getMiddle());
          data[index++] = intern(triple.getRight());
        }

View on GitHub (pinned to 2e990059ce)

Solutions

1. Increase OFFSET_SHIFT (widening the offset field) so MAX_OFFSET covers the value count
2. Reduce the size of the synthesizer data file by removing unused lemma/POS entries

When it happens

Trigger: Thrown at languagetool-core/src/main/java/org/languagetool/synthesis/ManualSynthesizer.java:77 when the library encounters an invalid state.

Common situations: See trigger scenarios.

AI-assisted analysis of languagetool-org/languagetool@2e990059ce (2026-09-06). Data as JSON: /api/errors/0f1fc779bf029d7c. Report an issue: GitHub.