{"record":{"id":"76a1d1a52a67d906","repo":"apache/druid","slug":"dictionary-not-serialized-cannot-open-value-seria","errorCode":null,"errorMessage":"Dictionary not serialized, cannot open value serializer","messagePattern":"Dictionary not serialized, cannot open value serializer","errorType":"exception","errorClass":"java.lang.IllegalStateException","httpStatus":null,"severity":"error","filePath":"processing/src/main/java/org/apache/druid/segment/nested/ScalarNestedCommonFormatColumnSerializer.java","lineNumber":133,"sourceCode":"  @Override\n  public void setDictionaryIdLookup(DictionaryIdLookup dictionaryIdLookup)\n  {\n    this.dictionaryIdLookup = dictionaryIdLookup;\n    this.writeDictionary = false;\n    this.dictionarySerialized = true;\n  }\n\n  @Override\n  public boolean hasNulls()\n  {\n    return hasNulls;\n  }\n\n  @Override\n  public void open() throws IOException\n  {\n    if (!dictionarySerialized) {\n      throw new IllegalStateException(\"Dictionary not serialized, cannot open value serializer\");\n    }\n    intermediateValueWriter = new FixedIndexedIntWriter(segmentWriteOutMedium, false);\n    intermediateValueWriter.open();\n    openValueColumnSerializer();\n  }\n\n  @Override\n  public void serialize(ColumnValueSelector<? extends StructuredData> selector) throws IOException\n  {\n    if (!dictionarySerialized) {\n      throw new ISE(\"Must serialize value dictionaries before serializing values for column [%s]\", name);\n    }\n\n    final Object value = StructuredData.unwrap(selector.getObject());\n    final int dictId = processValue(value);\n    intermediateValueWriter.write(dictId);\n    hasNulls = hasNulls || dictId == 0;\n  }","sourceCodeStart":115,"sourceCodeEnd":151,"githubUrl":"https://github.com/apache/druid/blob/9b90983fd291f26935af934383ce360473179e4d/processing/src/main/java/org/apache/druid/segment/nested/ScalarNestedCommonFormatColumnSerializer.java#L115-L151","documentation":"ScalarNestedCommonFormatColumnSerializer.open() creates the intermediate value writer, but only after the value dictionary has been serialized (dictionarySerialized flag). Opening the value serializer before the dictionary exists would leave the intermediate FixedIndexedIntWriter pointing at a dictionary that never gets written, so it throws IllegalStateException.","triggerScenarios":"Calling open() on the serializer before calling serializeDictionaries(); an ingestion pipeline that orders phases incorrectly (open -> serialize values -> serialize dictionaries instead of dictionaries first).","commonSituations":"Custom segment-creation code wiring the serializer lifecycle out of order; generic IndexMerger implementations that forget the nested-column dictionary phase before opening writers.","solutions":["Call serializeDictionaries() before calling open() so the dictionarySerialized flag is set","Fix the phase ordering in the segment-writing pipeline: write dictionaries, then open, then serialize values","If writing custom merger logic, mirror the ordering used by IndexMergerV9/nested-column support in druid-processing"],"exampleFix":"// before\nserializer.open();                               // ISE: dictionary not serialized\nserializer.serializeDictionaries(s, l, d, a);\n// after\nserializer.serializeDictionaries(s, l, d, a);\nserializer.open();\nserializer.serialize(selector);","handlingStrategy":"validation","validationCode":"// Enforce phase order in the writing pipeline before open()\nObjects.requireNonNull(writer.getDictionaryState(), \"dictionary state must exist\");\nif (!writer.isDictionarySerialized()) {\n  writer.serializeDictionaries(strings, longs, doubles, arrays); // run dictionary phase first\n}\nwriter.open();","typeGuard":null,"tryCatchPattern":"try {\n  serializer.open();\n} catch (IllegalStateException e) {\n  if (e.getMessage() != null && e.getMessage().contains(\"Dictionary not serialized\")) {\n    LOG.error(e, \"open() called before serializeDictionaries(); fixing phase order and restarting segment write\");\n    restartSegmentWriteFromScratch();\n  } else {\n    throw e;\n  }\n}","preventionTips":["Call serializer lifecycle methods strictly in order: serializeDictionaries -> open -> serialize","Centralize serializer lifecycle in one owner class so phases cannot be invoked out of order","Add debug assertions in custom pipelines that dictionary serialization completed before open()","Write a unit test that exercises out-of-order calls to verify failures are caught in CI, not production"],"tags":["segment-serialization","illegal-state","lifecycle-order","druid-processing"],"backgroundTag":"invalid-state-transition","analyzedSha":"9b90983fd291f26935af934383ce360473179e4d","analyzedAt":"2026-09-07T13:32:30.957Z","contentChangedAt":"2026-09-07T13:32:30.957Z","schemaVersion":2},"datasetVersion":"2026-09-17T15:17:12.973Z"}