apache/druid · error · org.apache.druid.java.util.common.ISE

Double dictionary already serialized for column

Error message

Double dictionary already serialized for column [%s], cannot serialize again

What it means

ScalarDoubleColumnSerializer.serializeDictionaries() guards an internal lifecycle flag: the double dictionary for a column may be written to the segment write-out medium exactly once. Calling it a second time on the same serializer instance would corrupt the segment (duplicate dictionary), so an IllegalStateException is thrown.

Solutions

Create a fresh ScalarDoubleColumnSerializer instance (via the column serializer factory) and restart serialization from scratch
Restructure the serialization pipeline so smooshify/serializeDictionaries is invoked exactly once per column
Discard the partially written segment file after any failure and rebuild the whole segment instead of retrying in place

Example fix

// before
serializer.serializeDictionaries(strings, longs, doubles, arrays);
serializer.serializeDictionaries(strings, longs, doubles, arrays); // ISE
// after
serializer.serializeDictionaries(strings, longs, doubles, arrays);
serializer.serializeColumns(); // proceed to next phase instead of re-serializing

Defensive patterns

Strategy: validation

Validate before calling

// Java-side guard in custom merge code before invoking the dictionary phase
if (isDictionarySerialized(serializer)) {
  throw new IllegalStateException("Skip serializeDictionaries; already done for this column");
}
serializer.serializeDictionaries(strings, longs, doubles, arrays);

Try / catch

try {
  serializer.serializeDictionaries(strings, longs, doubles, arrays);
} catch (IllegalStateException e) {
  if (e.getMessage() != null && e.getMessage().contains("already serialized")) {
    // Phase already ran; treat as no-op rather than retrying with the same instance
    LOG.warn(e, "Dictionary phase already completed");
  } else {
    throw e;
  }
}

Prevention

Treat serializer objects as single-use: create a new instance for every serialization attempt
Encode the pipeline as an explicit state machine (dictionaries -> open -> values) so phases cannot repeat
Never retry a failed segment write in place; rebuild the whole segment from source data
Add integration tests for custom merge code that exercise smooshify end-to-end once per column

When it happens

Trigger: Calling serializeDictionaries() twice on the same ScalarDoubleColumnSerializer instance, or a caller pipeline (e.g. smooshify) invoking the dictionary-serialization phase more than once per column.

Common situations: Custom ingestion/segment-merging code that retries serialization after a partial failure without recreating the serializer; miswired IndexMerger paths that run smooshify twice over the same column serializer.

Understand the failure class

Background: "Invalid state transition" errors: "status must be X, actually Y", "already rejected/charging/uninstalled", "cannot ... while running" — what they mean when a library rejects your call — this error's family across 31 libraries.

Related errors

AI-assisted analysis of apache/druid@9b90983fd2 (2026-09-07). Data as JSON: /api/errors/6d603c7cc1e88459. Report an issue: GitHub.

Appendix: source

Thrown at processing/src/main/java/org/apache/druid/segment/nested/ScalarDoubleColumnSerializer.java:113

        segmentWriteOutMedium,
        StringUtils.format("%s.double_column", name),
        ByteOrder.nativeOrder(),
        columnFormatSpec.getDoubleColumnCompression(),
        segmentWriteOutMedium.getCloser()
    );
    doublesSerializer.open();
  }

  @Override
  public void serializeDictionaries(
      Iterable<String> strings,
      Iterable<Long> longs,
      Iterable<Double> doubles,
      Iterable<int[]> arrays
  ) throws IOException
  {
    if (dictionarySerialized) {
      throw new ISE("Double dictionary already serialized for column [%s], cannot serialize again", name);
    }

    // null is always 0
    dictionaryWriter.write(null);

    for (Double value : doubles) {
      if (value == null) {
        continue;
      }
      dictionaryWriter.write(value);
    }
    dictionarySerialized = true;

  }

  @Override
  protected void writeValueColumn(SegmentFileBuilder fileBuilder) throws IOException
  {

View on GitHub (pinned to 9b90983fd2)