apache/druid · error · java.lang.IllegalStateException

Dictionary not serialized, cannot open value serializer

Error message

Dictionary not serialized, cannot open value serializer

What it means

VariantColumnSerializer.open() initializes the intermediate value writer, but this is only legal after the value dictionaries have been serialized (dictionarySerialized == true). Calling open() beforehand leaves the serializer in an unusable state, so it throws IllegalStateException 'Dictionary not serialized, cannot open value serializer'.

Solutions

1. Call serializeDictionaries() before open() in the writer lifecycle
2. Follow the standard serializer contract: open the dictionary writer, serialize dictionaries, then open the value writer
3. Upgrade Druid if a merge task triggers this through an internal bug
4. Review any custom ColumnSerializer subclass for out-of-order lifecycle calls

Example fix

// before
serializer.open();
serializer.serializeDictionaries(strings, longs, doubles, arrays);
// after
serializer.serializeDictionaries(strings, longs, doubles, arrays);
serializer.open();

if (!serializer.isDictionarySerialized()) { serializer.serializeDictionaries(strings, longs, doubles, arrays); } serializer.open();

try { serializer.open(); } catch (IllegalStateException e) { if (e.getMessage().contains("Dictionary not serialized")) { /* fix lifecycle order and retry with a fresh serializer */ } throw e; }

When it happens

Trigger: Calling open() before serializeDictionaries() during segment merge (smooshify), or a merge code path that invokes open out of order.

Common situations: Custom segment merge/compression tooling that calls the writer lifecycle methods in the wrong order; Druid internal bugs in segment merge tasks.

Understand the failure class

Background: "Invalid state transition" errors: "status must be X, actually Y", "already rejected/charging/uninstalled", "cannot ... while running" — what they mean when a library rejects your call — this error's family across 31 libraries.

Related errors

• Must serialize value dictionaries before serializing values…
• Value dictionaries already serialized for column
• Already started.
• Already started, not starting again
• can't stop.

AI-assisted analysis of apache/druid@9b90983fd2 (2026-09-07). Data as JSON: /api/errors/56d6e1d821aa2930. Report an issue: GitHub.

Appendix: source

Thrown at processing/src/main/java/org/apache/druid/segment/nested/VariantColumnSerializer.java:186

    arrayElementDictionaryWriter = new FixedIndexedIntWriter(segmentWriteOutMedium, true);
    arrayElementDictionaryWriter.open();
    dictionaryIdLookup = closer.register(
        new DictionaryIdLookup(
            name,
            segmentBaseDir,
            dictionaryWriter,
            longDictionaryWriter,
            doubleDictionaryWriter,
            arrayDictionaryWriter
        )
    );
  }

  @Override
  public void open() throws IOException
  {
    if (!dictionarySerialized) {
      throw new IllegalStateException("Dictionary not serialized, cannot open value serializer");
    }
    intermediateValueWriter = new FixedIndexedIntWriter(segmentWriteOutMedium, false);
    intermediateValueWriter.open();
  }

  @Override
  public void serializeDictionaries(
      Iterable<String> strings,
      Iterable<Long> longs,
      Iterable<Double> doubles,
      Iterable<int[]> arrays
  ) throws IOException
  {
    if (dictionarySerialized) {
      throw new ISE("Value dictionaries already serialized for column [%s], cannot serialize again", name);
    }

    // null is always 0

View on GitHub (pinned to 9b90983fd2)