apache/druid · error · org.apache.druid.java.util.common.ISE

Must serialize value dictionaries before serializing values…

Error message

Must serialize value dictionaries before serializing values for column [%s]

What it means

VariantColumnSerializer.serialize writes row values and relies on the global dictionary ids produced by serializeDictionaries. If dictionaries were not serialized first (dictionarySerialized == false), the value writer has no id space to reference, so serialize throws ISE 'Must serialize value dictionaries before serializing values'.

Solutions

1. Always call serializeDictionaries() before serialize() for the column
2. Verify the merge/ingestion task code path invokes both phases unconditionally
3. Recreate the writer with the correct lifecycle if a prior failure left it partially initialized
4. Upgrade Druid if an internal task path triggers this ordering bug

Example fix

// before
serializer.serialize(selector);
// after
serializer.serializeDictionaries(strings, longs, doubles, arrays);
serializer.open();
serializer.serialize(selector);

if (!serializer.isDictionarySerialized()) { throw new IllegalStateException("call serializeDictionaries before serialize for " + name); }

try { serializer.serialize(selector); } catch (ISE e) { if (e.getMessage().startsWith("Must serialize value dictionaries")) { /* reorder lifecycle and restart write */ } throw e; }

When it happens

Trigger: Calling serialize(ColumnValueSelector) before serializeDictionaries during a segment merge/ingestion write, or a selector-driven write path that skips the dictionary phase for a column with values.

Common situations: Custom ingestion/merge code invoking writer methods out of order; pipeline refactors that moved the dictionary phase behind a conditional that never runs.

Understand the failure class

Background: "Invalid state transition" errors: "status must be X, actually Y", "already rejected/charging/uninstalled", "cannot ... while running" — what they mean when a library rejects your call — this error's family across 31 libraries.

Related errors

• Dictionary not serialized, cannot open value serializer
• Value dictionaries already serialized for column
• Already started.
• Already started, not starting again
• can't stop.

AI-assisted analysis of apache/druid@9b90983fd2 (2026-09-07). Data as JSON: /api/errors/1f948c3c743bcd88. Report an issue: GitHub.

Appendix: source

Thrown at processing/src/main/java/org/apache/druid/segment/nested/VariantColumnSerializer.java:241

        continue;
      }
      doubleDictionaryWriter.write(value);
    }

    for (int[] value : arrays) {
      if (value == null) {
        continue;
      }
      arrayDictionaryWriter.write(value);
    }
    dictionarySerialized = true;
  }

  @Override
  public void serialize(ColumnValueSelector<? extends StructuredData> selector) throws IOException
  {
    if (!dictionarySerialized) {
      throw new ISE("Must serialize value dictionaries before serializing values for column [%s]", name);
    }

    ExprEval eval = ExprEval.bestEffortOf(StructuredData.unwrap(selector.getObject()));
    if (expectedExpressionType != null) {
      try {
        eval = eval.castTo(expectedExpressionType);
      }
      catch (IAE invalidCast) {
        // write null
        intermediateValueWriter.write(0);
        hasNulls = true;
        return;
      }
    }
    if (eval.isArray()) {
      Object[] array = eval.asArray();
      if (array == null) {
        intermediateValueWriter.write(0);

View on GitHub (pinned to 9b90983fd2)