{"record":{"id":"b53d499bfacee3b3","repo":"apache/druid","slug":"failed-to-deserialize-v-s-column-b53d49","errorCode":null,"errorMessage":"Failed to deserialize V%s column.","messagePattern":"Failed to deserialize V(.+?) column\\.","errorType":"exception","errorClass":"org.apache.druid.java.util.common.RE","httpStatus":null,"severity":"error","filePath":"processing/src/main/java/org/apache/druid/segment/nested/ScalarLongColumnAndIndexSupplier.java","lineNumber":168,"sourceCode":"            columnBuilder.getFileMapper()\n        );\n\n        final Supplier<ColumnarLongs> longs = CompressedColumnarLongsSupplier.fromByteBuffer(\n            longsValueColumn,\n            byteOrder,\n            columnBuilder.getFileMapper()\n        );\n        return new ScalarLongColumnAndIndexSupplier(\n            longDictionarySupplier,\n            encodedCol,\n            longs,\n            rBitmaps,\n            bitmapSerdeFactory.getBitmapFactory(),\n            columnConfig\n        );\n      }\n      catch (IOException ex) {\n        throw new RE(ex, \"Failed to deserialize V%s column.\", version);\n      }\n    } else {\n      throw new RE(\"Unknown version \" + version);\n    }\n  }\n\n  private final Supplier<FixedIndexed<Long>> longDictionarySupplier;\n\n  private final Supplier<ColumnarInts> encodedValuesSupplier;\n  private final Supplier<ColumnarLongs> valueColumnSupplier;\n\n  private final GenericIndexed<ImmutableBitmap> valueIndexes;\n\n  private final BitmapFactory bitmapFactory;\n\n  private final ImmutableBitmap nullValueBitmap;\n  private final ColumnConfig columnConfig;\n","sourceCodeStart":150,"sourceCodeEnd":186,"githubUrl":"https://github.com/apache/druid/blob/9b90983fd291f26935af934383ce360473179e4d/processing/src/main/java/org/apache/druid/segment/nested/ScalarLongColumnAndIndexSupplier.java#L150-L186","documentation":"ScalarLongColumnAndIndexSupplier.read() wraps any IOException raised while deserializing a nested long column (dictionary, encoded values, or bitmaps) in this RuntimeException, attaching the column version. It means the on-disk bytes for the column could not be parsed — the file is truncated, corrupt, or was not written in the expected format.","triggerScenarios":"An IOException from reading the column part files: truncated/truncated-smooshed segment files, checksum mismatch, partially written segment from a crashed merge, or incompatible binary layout for the declared version.","commonSituations":"Disk corruption or incomplete segment replication on historicals; segments copied from deep storage mid-upload; running out of disk space during segment write leaving truncated files that are later read.","solutions":["Replace the corrupt segment with an intact replica (or re-download from deep storage)","Re-ingest the affected time chunk to regenerate the segment","Check historical/deep-storage disk health and available space; repair underlying storage issues","If reproducible after a version change, re-serialize segments with the current Druid version"],"exampleFix":null,"handlingStrategy":"try-catch","validationCode":"// Pre-check segment files before loading: sizes and readability of column part files\nfor (File part : segmentDir.listFiles()) {\n  if (part.length() == 0 || !part.canRead()) {\n    throw new IOException(\"Segment file incomplete/unreadable: \" + part);\n  }\n}","typeGuard":null,"tryCatchPattern":"try {\n  ColumnHolder col = NestedDataColumnAndIndexSupplier.read(...);\n  // use long column\n} catch (RuntimeException e) {\n  if (e.getCause() instanceof IOException) {\n    LOG.error(e, \"Corrupt long column in %s; falling back to replica/re-ingest\", segmentId);\n    segmentLoader.drop(segmentId);\n    replicas.reroute(segmentId);\n  } else {\n    throw e;\n  }\n}","preventionTips":["Enable checksum validation of segment files when copying between deep storage and cache","Ensure enough disk headroom during merges so segments are never written partially","Verify successful segment upload before deleting local staging data","Monitor historical logs for I/O errors and fail fast on suspect storage devices"],"tags":["segment-deserialization","ioexception","data-corruption","druid-processing"],"backgroundTag":"file-read-failed","analyzedSha":"9b90983fd291f26935af934383ce360473179e4d","analyzedAt":"2026-09-07T13:32:30.957Z","contentChangedAt":"2026-09-07T13:32:30.957Z","schemaVersion":2},"datasetVersion":"2026-09-17T15:17:12.973Z"}