{"record":{"id":"bad0e260b7853a8f","repo":"apache/druid","slug":"long-dictionary-already-serialized-for-column-s","errorCode":null,"errorMessage":"Long dictionary already serialized for column [%s], cannot serialize again","messagePattern":"Long dictionary already serialized for column \\[(.+?)\\], cannot serialize again","errorType":"exception","errorClass":"org.apache.druid.java.util.common.ISE","httpStatus":null,"severity":"error","filePath":"processing/src/main/java/org/apache/druid/segment/nested/ScalarLongColumnSerializer.java","lineNumber":115,"sourceCode":"        StringUtils.format(\"%s.long_column\", name),\n        ByteOrder.nativeOrder(),\n        columnFormatSpec.getLongColumnEncoding(),\n        columnFormatSpec.getLongColumnCompression(),\n        segmentWriteOutMedium.getCloser()\n    );\n    longsSerializer.open();\n  }\n\n  @Override\n  public void serializeDictionaries(\n      Iterable<String> strings,\n      Iterable<Long> longs,\n      Iterable<Double> doubles,\n      Iterable<int[]> arrays\n  ) throws IOException\n  {\n    if (dictionarySerialized) {\n      throw new ISE(\"Long dictionary already serialized for column [%s], cannot serialize again\", name);\n    }\n\n    // null is always 0\n    dictionaryWriter.write(null);\n    for (Long value : longs) {\n      if (value == null) {\n        continue;\n      }\n      dictionaryWriter.write(value);\n    }\n    dictionarySerialized = true;\n  }\n\n  @Override\n  protected void writeValueColumn(SegmentFileBuilder fileBuilder) throws IOException\n  {\n    writeInternal(fileBuilder, longsSerializer, ColumnSerializerUtils.LONG_VALUE_COLUMN_FILE_NAME);\n  }","sourceCodeStart":97,"sourceCodeEnd":133,"githubUrl":"https://github.com/apache/druid/blob/9b90983fd291f26935af934383ce360473179e4d/processing/src/main/java/org/apache/druid/segment/nested/ScalarLongColumnSerializer.java#L97-L133","documentation":"ScalarLongColumnSerializer.serializeDictionaries() enforces a one-shot lifecycle: once the long dictionary has been written to the segment write-out medium, a second call would produce a duplicate/corrupt dictionary, so it throws IllegalStateException guarded by the dictionarySerialized flag.","triggerScenarios":"Invoking serializeDictionaries() twice on the same ScalarLongColumnSerializer instance, or the smooshify pipeline re-entering the dictionary phase for the same column after it already completed.","commonSituations":"Custom merger/compaction code that retries after a partial failure without resetting the serializer; accidentally running smooshify twice over one column serializer in custom ingestion pipelines.","solutions":["Instantiate a fresh ScalarLongColumnSerializer and restart the column serialization from scratch","Ensure the merge pipeline invokes serializeDictionaries exactly once per column before serializing values","After any serialization failure, discard the partially written segment and rebuild it entirely rather than retrying in place"],"exampleFix":"// before\nlongSerializer.serializeDictionaries(strings, longs, doubles, arrays);\nlongSerializer.serializeDictionaries(strings, longs, doubles, arrays); // ISE\n// after\nlongSerializer.serializeDictionaries(strings, longs, doubles, arrays);\nlongSerializer.serializeColumns();","handlingStrategy":"validation","validationCode":"// Guard before entering the dictionary phase in custom merge code\nif (isDictionarySerialized(longSerializer)) {\n  throw new IllegalStateException(\"Long dictionary phase already ran for this column\");\n}\nlongSerializer.serializeDictionaries(strings, longs, doubles, arrays);","typeGuard":null,"tryCatchPattern":"try {\n  longSerializer.serializeDictionaries(strings, longs, doubles, arrays);\n} catch (IllegalStateException e) {\n  if (e.getMessage() != null && e.getMessage().contains(\"already serialized\")) {\n    LOG.warn(e, \"Long dictionary already written; skipping duplicate phase\");\n  } else {\n    throw e;\n  }\n}","preventionTips":["Instantiate a fresh serializer per merge attempt instead of reusing one across retries","Document and enforce the phase order (serializeDictionaries -> serializeColumns) in custom pipelines","Fail fast and rebuild from source on any write failure rather than resuming mid-segment","Test custom merger paths with a once-only assertion around each serializer phase"],"tags":["segment-serialization","illegal-state","druid-processing"],"backgroundTag":"invalid-state-transition","analyzedSha":"9b90983fd291f26935af934383ce360473179e4d","analyzedAt":"2026-09-07T13:32:30.957Z","contentChangedAt":"2026-09-07T13:32:30.957Z","schemaVersion":2},"datasetVersion":"2026-09-17T15:17:12.973Z"}