apache/druid · error · org.apache.druid.java.util.common.ISE

String dictionary already serialized for column

Error message

String dictionary already serialized for column [%s], cannot serialize again

What it means

ScalarStringColumnSerializer.serializeDictionaries() is a one-shot phase guarded by the dictionarySerialized flag: once the string dictionary has been written (with null as id 0), re-serializing it would duplicate the dictionary and corrupt the segment, so IllegalStateException is thrown.

Solutions

Create a new ScalarStringColumnSerializer instance and restart the column serialization from the beginning
Ensure the pipeline calls serializeDictionaries exactly once per column, before value serialization
Discard partially written segment output after a failure and rebuild the entire segment instead of retrying in place

Example fix

// before
stringSerializer.serializeDictionaries(strings, longs, doubles, arrays);
stringSerializer.serializeDictionaries(strings, longs, doubles, arrays); // ISE
// after
stringSerializer.serializeDictionaries(strings, longs, doubles, arrays);
stringSerializer.serializeColumns();

Defensive patterns

Strategy: validation

Validate before calling

// Custom pipeline: assert the string dictionary phase has not run before invoking it
if (isDictionarySerialized(stringSerializer)) {
  throw new IllegalStateException("String dictionary phase already completed for this column");
}
stringSerializer.serializeDictionaries(strings, longs, doubles, arrays);

Try / catch

try {
  stringSerializer.serializeDictionaries(strings, longs, doubles, arrays);
} catch (IllegalStateException e) {
  if (e.getMessage() != null && e.getMessage().contains("already serialized")) {
    LOG.warn(e, "String dictionary already written; skipping duplicate call");
  } else {
    throw e;
  }
}

Prevention

Make serializer instances strictly single-use; recreate per merge attempt
Enforce a single invocation of serializeDictionaries per column with a pipeline-level guard flag
On any serialization failure, rebuild the segment from source rather than retrying with the same serializer
Add end-to-end tests of custom smooshify flows to confirm each phase runs exactly once

When it happens

Trigger: Calling serializeDictionaries() twice on the same ScalarStringColumnSerializer instance, or the smooshify merge pipeline entering the dictionary-serialization phase twice for one column.

Common situations: Retry logic in custom segment-merging code that reuses a serializer instance after failure; duplicate invocation of smooshify over the same column serializer in bespoke ingestion pipelines.

Understand the failure class

Background: "Invalid state transition" errors: "status must be X, actually Y", "already rejected/charging/uninstalled", "cannot ... while running" — what they mean when a library rejects your call — this error's family across 31 libraries.

Related errors

AI-assisted analysis of apache/druid@9b90983fd2 (2026-09-07). Data as JSON: /api/errors/010424e19a215f24. Report an issue: GitHub.

Appendix: source

Thrown at processing/src/main/java/org/apache/druid/segment/nested/ScalarStringColumnSerializer.java:98

    );
  }

  @Override
  protected void openValueColumnSerializer()
  {
    // no extra value column for strings
  }

  @Override
  public void serializeDictionaries(
      Iterable<String> strings,
      Iterable<Long> longs,
      Iterable<Double> doubles,
      Iterable<int[]> arrays
  ) throws IOException
  {
    if (dictionarySerialized) {
      throw new ISE("String dictionary already serialized for column [%s], cannot serialize again", name);
    }

    // null is always 0
    dictionaryWriter.write(null);
    for (String value : strings) {
      if (value == null) {
        continue;
      }

      dictionaryWriter.write(value);
    }
    dictionarySerialized = true;
  }

  @Override
  protected void writeValueColumn(SegmentFileBuilder fileBuilder)
  {
    // no extra value column for strings

View on GitHub (pinned to 9b90983fd2)