{"record":{"id":"6d603c7cc1e88459","repo":"apache/druid","slug":"double-dictionary-already-serialized-for-column","errorCode":null,"errorMessage":"Double dictionary already serialized for column [%s], cannot serialize again","messagePattern":"Double dictionary already serialized for column \\[(.+?)\\], cannot serialize again","errorType":"exception","errorClass":"org.apache.druid.java.util.common.ISE","httpStatus":null,"severity":"error","filePath":"processing/src/main/java/org/apache/druid/segment/nested/ScalarDoubleColumnSerializer.java","lineNumber":113,"sourceCode":"        segmentWriteOutMedium,\n        StringUtils.format(\"%s.double_column\", name),\n        ByteOrder.nativeOrder(),\n        columnFormatSpec.getDoubleColumnCompression(),\n        segmentWriteOutMedium.getCloser()\n    );\n    doublesSerializer.open();\n  }\n\n  @Override\n  public void serializeDictionaries(\n      Iterable<String> strings,\n      Iterable<Long> longs,\n      Iterable<Double> doubles,\n      Iterable<int[]> arrays\n  ) throws IOException\n  {\n    if (dictionarySerialized) {\n      throw new ISE(\"Double dictionary already serialized for column [%s], cannot serialize again\", name);\n    }\n\n    // null is always 0\n    dictionaryWriter.write(null);\n\n    for (Double value : doubles) {\n      if (value == null) {\n        continue;\n      }\n      dictionaryWriter.write(value);\n    }\n    dictionarySerialized = true;\n\n  }\n\n  @Override\n  protected void writeValueColumn(SegmentFileBuilder fileBuilder) throws IOException\n  {","sourceCodeStart":95,"sourceCodeEnd":131,"githubUrl":"https://github.com/apache/druid/blob/9b90983fd291f26935af934383ce360473179e4d/processing/src/main/java/org/apache/druid/segment/nested/ScalarDoubleColumnSerializer.java#L95-L131","documentation":"ScalarDoubleColumnSerializer.serializeDictionaries() guards an internal lifecycle flag: the double dictionary for a column may be written to the segment write-out medium exactly once. Calling it a second time on the same serializer instance would corrupt the segment (duplicate dictionary), so an IllegalStateException is thrown.","triggerScenarios":"Calling serializeDictionaries() twice on the same ScalarDoubleColumnSerializer instance, or a caller pipeline (e.g. smooshify) invoking the dictionary-serialization phase more than once per column.","commonSituations":"Custom ingestion/segment-merging code that retries serialization after a partial failure without recreating the serializer; miswired IndexMerger paths that run smooshify twice over the same column serializer.","solutions":["Create a fresh ScalarDoubleColumnSerializer instance (via the column serializer factory) and restart serialization from scratch","Restructure the serialization pipeline so smooshify/serializeDictionaries is invoked exactly once per column","Discard the partially written segment file after any failure and rebuild the whole segment instead of retrying in place"],"exampleFix":"// before\nserializer.serializeDictionaries(strings, longs, doubles, arrays);\nserializer.serializeDictionaries(strings, longs, doubles, arrays); // ISE\n// after\nserializer.serializeDictionaries(strings, longs, doubles, arrays);\nserializer.serializeColumns(); // proceed to next phase instead of re-serializing","handlingStrategy":"validation","validationCode":"// Java-side guard in custom merge code before invoking the dictionary phase\nif (isDictionarySerialized(serializer)) {\n  throw new IllegalStateException(\"Skip serializeDictionaries; already done for this column\");\n}\nserializer.serializeDictionaries(strings, longs, doubles, arrays);","typeGuard":null,"tryCatchPattern":"try {\n  serializer.serializeDictionaries(strings, longs, doubles, arrays);\n} catch (IllegalStateException e) {\n  if (e.getMessage() != null && e.getMessage().contains(\"already serialized\")) {\n    // Phase already ran; treat as no-op rather than retrying with the same instance\n    LOG.warn(e, \"Dictionary phase already completed\");\n  } else {\n    throw e;\n  }\n}","preventionTips":["Treat serializer objects as single-use: create a new instance for every serialization attempt","Encode the pipeline as an explicit state machine (dictionaries -> open -> values) so phases cannot repeat","Never retry a failed segment write in place; rebuild the whole segment from source data","Add integration tests for custom merge code that exercise smooshify end-to-end once per column"],"tags":["segment-serialization","illegal-state","druid-processing"],"backgroundTag":"invalid-state-transition","analyzedSha":"9b90983fd291f26935af934383ce360473179e4d","analyzedAt":"2026-09-07T13:32:30.957Z","contentChangedAt":"2026-09-07T13:32:30.957Z","schemaVersion":2},"datasetVersion":"2026-09-17T15:17:12.973Z"}