{"record":{"id":"1f948c3c743bcd88","repo":"apache/druid","slug":"must-serialize-value-dictionaries-before-serializi-1f948c","errorCode":null,"errorMessage":"Must serialize value dictionaries before serializing values for column [%s]","messagePattern":"Must serialize value dictionaries before serializing values for column \\[(.+?)\\]","errorType":"exception","errorClass":"org.apache.druid.java.util.common.ISE","httpStatus":null,"severity":"error","filePath":"processing/src/main/java/org/apache/druid/segment/nested/VariantColumnSerializer.java","lineNumber":241,"sourceCode":"        continue;\n      }\n      doubleDictionaryWriter.write(value);\n    }\n\n    for (int[] value : arrays) {\n      if (value == null) {\n        continue;\n      }\n      arrayDictionaryWriter.write(value);\n    }\n    dictionarySerialized = true;\n  }\n\n  @Override\n  public void serialize(ColumnValueSelector<? extends StructuredData> selector) throws IOException\n  {\n    if (!dictionarySerialized) {\n      throw new ISE(\"Must serialize value dictionaries before serializing values for column [%s]\", name);\n    }\n\n    ExprEval eval = ExprEval.bestEffortOf(StructuredData.unwrap(selector.getObject()));\n    if (expectedExpressionType != null) {\n      try {\n        eval = eval.castTo(expectedExpressionType);\n      }\n      catch (IAE invalidCast) {\n        // write null\n        intermediateValueWriter.write(0);\n        hasNulls = true;\n        return;\n      }\n    }\n    if (eval.isArray()) {\n      Object[] array = eval.asArray();\n      if (array == null) {\n        intermediateValueWriter.write(0);","sourceCodeStart":223,"sourceCodeEnd":259,"githubUrl":"https://github.com/apache/druid/blob/9b90983fd291f26935af934383ce360473179e4d/processing/src/main/java/org/apache/druid/segment/nested/VariantColumnSerializer.java#L223-L259","documentation":"VariantColumnSerializer.serialize writes row values and relies on the global dictionary ids produced by serializeDictionaries. If dictionaries were not serialized first (dictionarySerialized == false), the value writer has no id space to reference, so serialize throws ISE 'Must serialize value dictionaries before serializing values'.","triggerScenarios":"Calling serialize(ColumnValueSelector) before serializeDictionaries during a segment merge/ingestion write, or a selector-driven write path that skips the dictionary phase for a column with values.","commonSituations":"Custom ingestion/merge code invoking writer methods out of order; pipeline refactors that moved the dictionary phase behind a conditional that never runs.","solutions":["Always call serializeDictionaries() before serialize() for the column","Verify the merge/ingestion task code path invokes both phases unconditionally","Recreate the writer with the correct lifecycle if a prior failure left it partially initialized","Upgrade Druid if an internal task path triggers this ordering bug"],"exampleFix":"// before\nserializer.serialize(selector);\n// after\nserializer.serializeDictionaries(strings, longs, doubles, arrays);\nserializer.open();\nserializer.serialize(selector);","handlingStrategy":"validation","validationCode":"if (!serializer.isDictionarySerialized()) { throw new IllegalStateException(\"call serializeDictionaries before serialize for \" + name); }","typeGuard":null,"tryCatchPattern":"try { serializer.serialize(selector); } catch (ISE e) { if (e.getMessage().startsWith(\"Must serialize value dictionaries\")) { /* reorder lifecycle and restart write */ } throw e; }","preventionTips":["Enforce phase ordering in a single merge orchestration method","Assert dictionarySerialized before value writes in custom tooling","Test merge pipelines against segment-compression and migration tasks"],"tags":["druid","segment-writing","lifecycle"],"backgroundTag":"invalid-state-transition","analyzedSha":"9b90983fd291f26935af934383ce360473179e4d","analyzedAt":"2026-09-07T13:32:30.957Z","contentChangedAt":"2026-09-07T13:32:30.957Z","schemaVersion":2},"datasetVersion":"2026-09-17T15:17:12.973Z"}