apache/druid · error · org.apache.druid.java.util.common.ISE

Long dictionary already serialized for column

Error message

Long dictionary already serialized for column [%s], cannot serialize again

What it means

ScalarLongColumnSerializer.serializeDictionaries() enforces a one-shot lifecycle: once the long dictionary has been written to the segment write-out medium, a second call would produce a duplicate/corrupt dictionary, so it throws IllegalStateException guarded by the dictionarySerialized flag.

Solutions

Instantiate a fresh ScalarLongColumnSerializer and restart the column serialization from scratch
Ensure the merge pipeline invokes serializeDictionaries exactly once per column before serializing values
After any serialization failure, discard the partially written segment and rebuild it entirely rather than retrying in place

Example fix

// before
longSerializer.serializeDictionaries(strings, longs, doubles, arrays);
longSerializer.serializeDictionaries(strings, longs, doubles, arrays); // ISE
// after
longSerializer.serializeDictionaries(strings, longs, doubles, arrays);
longSerializer.serializeColumns();

Defensive patterns

Strategy: validation

Validate before calling

// Guard before entering the dictionary phase in custom merge code
if (isDictionarySerialized(longSerializer)) {
  throw new IllegalStateException("Long dictionary phase already ran for this column");
}
longSerializer.serializeDictionaries(strings, longs, doubles, arrays);

Try / catch

try {
  longSerializer.serializeDictionaries(strings, longs, doubles, arrays);
} catch (IllegalStateException e) {
  if (e.getMessage() != null && e.getMessage().contains("already serialized")) {
    LOG.warn(e, "Long dictionary already written; skipping duplicate phase");
  } else {
    throw e;
  }
}

Prevention

Instantiate a fresh serializer per merge attempt instead of reusing one across retries
Document and enforce the phase order (serializeDictionaries -> serializeColumns) in custom pipelines
Fail fast and rebuild from source on any write failure rather than resuming mid-segment
Test custom merger paths with a once-only assertion around each serializer phase

When it happens

Trigger: Invoking serializeDictionaries() twice on the same ScalarLongColumnSerializer instance, or the smooshify pipeline re-entering the dictionary phase for the same column after it already completed.

Common situations: Custom merger/compaction code that retries after a partial failure without resetting the serializer; accidentally running smooshify twice over one column serializer in custom ingestion pipelines.

Understand the failure class

Background: "Invalid state transition" errors: "status must be X, actually Y", "already rejected/charging/uninstalled", "cannot ... while running" — what they mean when a library rejects your call — this error's family across 31 libraries.

Related errors

AI-assisted analysis of apache/druid@9b90983fd2 (2026-09-07). Data as JSON: /api/errors/bad0e260b7853a8f. Report an issue: GitHub.

Appendix: source

Thrown at processing/src/main/java/org/apache/druid/segment/nested/ScalarLongColumnSerializer.java:115

        StringUtils.format("%s.long_column", name),
        ByteOrder.nativeOrder(),
        columnFormatSpec.getLongColumnEncoding(),
        columnFormatSpec.getLongColumnCompression(),
        segmentWriteOutMedium.getCloser()
    );
    longsSerializer.open();
  }

  @Override
  public void serializeDictionaries(
      Iterable<String> strings,
      Iterable<Long> longs,
      Iterable<Double> doubles,
      Iterable<int[]> arrays
  ) throws IOException
  {
    if (dictionarySerialized) {
      throw new ISE("Long dictionary already serialized for column [%s], cannot serialize again", name);
    }

    // null is always 0
    dictionaryWriter.write(null);
    for (Long value : longs) {
      if (value == null) {
        continue;
      }
      dictionaryWriter.write(value);
    }
    dictionarySerialized = true;
  }

  @Override
  protected void writeValueColumn(SegmentFileBuilder fileBuilder) throws IOException
  {
    writeInternal(fileBuilder, longsSerializer, ColumnSerializerUtils.LONG_VALUE_COLUMN_FILE_NAME);
  }

View on GitHub (pinned to 9b90983fd2)