{"record":{"id":"010424e19a215f24","repo":"apache/druid","slug":"string-dictionary-already-serialized-for-column","errorCode":null,"errorMessage":"String dictionary already serialized for column [%s], cannot serialize again","messagePattern":"String dictionary already serialized for column \\[(.+?)\\], cannot serialize again","errorType":"exception","errorClass":"org.apache.druid.java.util.common.ISE","httpStatus":null,"severity":"error","filePath":"processing/src/main/java/org/apache/druid/segment/nested/ScalarStringColumnSerializer.java","lineNumber":98,"sourceCode":"    );\n  }\n\n  @Override\n  protected void openValueColumnSerializer()\n  {\n    // no extra value column for strings\n  }\n\n  @Override\n  public void serializeDictionaries(\n      Iterable<String> strings,\n      Iterable<Long> longs,\n      Iterable<Double> doubles,\n      Iterable<int[]> arrays\n  ) throws IOException\n  {\n    if (dictionarySerialized) {\n      throw new ISE(\"String dictionary already serialized for column [%s], cannot serialize again\", name);\n    }\n\n    // null is always 0\n    dictionaryWriter.write(null);\n    for (String value : strings) {\n      if (value == null) {\n        continue;\n      }\n\n      dictionaryWriter.write(value);\n    }\n    dictionarySerialized = true;\n  }\n\n  @Override\n  protected void writeValueColumn(SegmentFileBuilder fileBuilder)\n  {\n    // no extra value column for strings","sourceCodeStart":80,"sourceCodeEnd":116,"githubUrl":"https://github.com/apache/druid/blob/9b90983fd291f26935af934383ce360473179e4d/processing/src/main/java/org/apache/druid/segment/nested/ScalarStringColumnSerializer.java#L80-L116","documentation":"ScalarStringColumnSerializer.serializeDictionaries() is a one-shot phase guarded by the dictionarySerialized flag: once the string dictionary has been written (with null as id 0), re-serializing it would duplicate the dictionary and corrupt the segment, so IllegalStateException is thrown.","triggerScenarios":"Calling serializeDictionaries() twice on the same ScalarStringColumnSerializer instance, or the smooshify merge pipeline entering the dictionary-serialization phase twice for one column.","commonSituations":"Retry logic in custom segment-merging code that reuses a serializer instance after failure; duplicate invocation of smooshify over the same column serializer in bespoke ingestion pipelines.","solutions":["Create a new ScalarStringColumnSerializer instance and restart the column serialization from the beginning","Ensure the pipeline calls serializeDictionaries exactly once per column, before value serialization","Discard partially written segment output after a failure and rebuild the entire segment instead of retrying in place"],"exampleFix":"// before\nstringSerializer.serializeDictionaries(strings, longs, doubles, arrays);\nstringSerializer.serializeDictionaries(strings, longs, doubles, arrays); // ISE\n// after\nstringSerializer.serializeDictionaries(strings, longs, doubles, arrays);\nstringSerializer.serializeColumns();","handlingStrategy":"validation","validationCode":"// Custom pipeline: assert the string dictionary phase has not run before invoking it\nif (isDictionarySerialized(stringSerializer)) {\n  throw new IllegalStateException(\"String dictionary phase already completed for this column\");\n}\nstringSerializer.serializeDictionaries(strings, longs, doubles, arrays);","typeGuard":null,"tryCatchPattern":"try {\n  stringSerializer.serializeDictionaries(strings, longs, doubles, arrays);\n} catch (IllegalStateException e) {\n  if (e.getMessage() != null && e.getMessage().contains(\"already serialized\")) {\n    LOG.warn(e, \"String dictionary already written; skipping duplicate call\");\n  } else {\n    throw e;\n  }\n}","preventionTips":["Make serializer instances strictly single-use; recreate per merge attempt","Enforce a single invocation of serializeDictionaries per column with a pipeline-level guard flag","On any serialization failure, rebuild the segment from source rather than retrying with the same serializer","Add end-to-end tests of custom smooshify flows to confirm each phase runs exactly once"],"tags":["segment-serialization","illegal-state","druid-processing"],"backgroundTag":"invalid-state-transition","analyzedSha":"9b90983fd291f26935af934383ce360473179e4d","analyzedAt":"2026-09-07T13:32:30.957Z","contentChangedAt":"2026-09-07T13:32:30.957Z","schemaVersion":2},"datasetVersion":"2026-09-17T15:17:12.973Z"}