{"record":{"id":"50dabb158139ad43","repo":"apache/druid","slug":"failed-to-deserialize-v-s-column-50dabb","errorCode":null,"errorMessage":"Failed to deserialize V%s column.","messagePattern":"Failed to deserialize V(.+?) column\\.","errorType":"exception","errorClass":"org.apache.druid.java.util.common.RE","httpStatus":null,"severity":"error","filePath":"processing/src/main/java/org/apache/druid/segment/nested/ScalarStringColumnAndIndexSupplier.java","lineNumber":109,"sourceCode":"        final ByteBuffer valueIndexBuffer = NestedCommonFormatColumnPartSerde.loadInternalFile(\n            mapper,\n            columnName,\n            ColumnSerializerUtils.BITMAP_INDEX_FILE_NAME\n        );\n        GenericIndexed<ImmutableBitmap> valueIndexes = GenericIndexed.read(\n            valueIndexBuffer,\n            bitmapSerdeFactory.getObjectStrategy(),\n            columnBuilder.getFileMapper()\n        );\n        return new ScalarStringColumnAndIndexSupplier(\n            dictionarySupplier,\n            ints,\n            valueIndexes,\n            bitmapSerdeFactory\n        );\n      }\n      catch (IOException ex) {\n        throw new RE(ex, \"Failed to deserialize V%s column.\", version);\n      }\n    } else {\n      throw new RE(\"Unknown version \" + version);\n    }\n  }\n\n\n  private final Supplier<? extends Indexed<ByteBuffer>> dictionarySupplier;\n  private final Supplier<ColumnarInts> encodedColumnSupplier;\n  private final GenericIndexed<ImmutableBitmap> valueIndexes;\n  private final ColumnIndexSupplier stringIndexSupplier;\n  private final BitmapSerdeFactory serdeFactory;\n\n  private ScalarStringColumnAndIndexSupplier(\n      Supplier<? extends Indexed<ByteBuffer>> dictionarySupplier,\n      Supplier<ColumnarInts> encodedColumnSupplier,\n      GenericIndexed<ImmutableBitmap> valueIndexes,\n      BitmapSerdeFactory serdeFactory","sourceCodeStart":91,"sourceCodeEnd":127,"githubUrl":"https://github.com/apache/druid/blob/9b90983fd291f26935af934383ce360473179e4d/processing/src/main/java/org/apache/druid/segment/nested/ScalarStringColumnAndIndexSupplier.java#L91-L127","documentation":"ScalarStringColumnAndIndexSupplier.read() catches IOException thrown while deserializing a nested string column (dictionary, encoded values, or value indexes) and rethrows it as this RuntimeException with the column version. It indicates the column's on-disk bytes could not be parsed.","triggerScenarios":"IOException during column part-file reads: truncated or corrupt segment files, failed checksum, partially written segment from a crashed merge/compaction, or unreadable cache/deep-storage copy.","commonSituations":"Hardware/storage failures corrupting historical segment directories; interrupted segment push leaving partial data in deep storage that is later read; cache holding a truncated segment.","solutions":["Drop the affected segment from the historical/cache and re-download it from deep storage","Re-ingest or re-compact the affected time chunk to rebuild the segment","Check storage health (disk errors, free space) and validate segment files against deep-storage checksums","If it follows a Druid upgrade, re-serialize segments with the new version's writer"],"exampleFix":null,"handlingStrategy":"try-catch","validationCode":"// Validate segment directory completeness before load\nfor (String required : new String[]{\"meta.smoosh\", \"00000.smoosh\", \"factory.json\"}) {\n  File f = new File(segmentDir, required);\n  if (!f.isFile() || f.length() == 0) {\n    throw new IOException(\"Segment missing/incomplete part: \" + f);\n  }\n}","typeGuard":null,"tryCatchPattern":"try {\n  // load nested string column indexes\n} catch (RuntimeException e) {\n  if (e.getCause() instanceof IOException) {\n    LOG.error(e, \"Failed to read string column of %s; dropping cache entry and rerouting\", segmentId);\n    segmentCache.drop(segmentId);\n    queryScheduler.reroute(segmentId);\n  } else {\n    throw e;\n  }\n}","preventionTips":["Verify deep-storage uploads complete (checksums) before marking segments as published","Keep adequate disk space on historicals to avoid truncated writes during merge","Periodically scrub deep storage for segment files that fail integrity checks","Clean stale cache entries after abnormal broker/historical shutdowns"],"tags":["segment-deserialization","ioexception","data-corruption","druid-processing"],"backgroundTag":"file-read-failed","analyzedSha":"9b90983fd291f26935af934383ce360473179e4d","analyzedAt":"2026-09-07T13:32:30.957Z","contentChangedAt":"2026-09-07T13:32:30.957Z","schemaVersion":2},"datasetVersion":"2026-09-17T15:17:12.973Z"}