apache/druid · error · RE

Failed to deserialize V%s column.

Error message

Failed to deserialize V%s column.

What it means

NestedDataColumnSupplier.read() deserializes a nested (auto) column's serialized state. When the underlying read throws an IOException, it is wrapped in a Druid RE with 'Failed to deserialize V<version> column.', indicating the on-disk column bytes could not be parsed for that column format version.

Source

Thrown at processing/src/main/java/org/apache/druid/segment/nested/NestedDataColumnSupplier.java:191

        return new NestedDataColumnSupplier(
            columnName,
            fieldsSupplier,
            fieldInfo,
            compressedRawColumnSupplier,
            nullValues,
            stringDictionarySupplier,
            longDictionarySupplier,
            doubleDictionarySupplier,
            arrayDictionarySupplier,
            columnConfig,
            mapper,
            formatSpec,
            byteOrder,
            logicalType
        );
      }
      catch (IOException ex) {
        throw new RE(ex, "Failed to deserialize V%s column.", version);
      }
    } else {
      throw new RE("Unknown version " + version);
    }
  }


  /**
   * Detects if field dictionary contains any invalid entries from a bug which previously existed in
   * {@link NestedPathFinder#toNormalizedJsonPath(List)} to generate invalid path expressions when faced with empty
   * field names - for example {"":{"a":1}} would incorrectly store the path as $..a instead of $[''].a.
   * <p>
   * If this method detects any illegal paths, the field dictionary is wrapped using {@link FieldsFixupIndexed}, which
   * replaces the invalid values with corrected values, using {@link NestedPathFinder#parseBadJsonPath(String)} and
   * feeding that back into the now fixed {@link NestedPathFinder#toNormalizedJsonPath(List)}.
   * <p>
   * Columns written after the bug was fixed will store {@link NestedCommonFormatColumnPartSerde#pathParserVersion} as
   * 0x01 or greater, to indicate that we do not need to call this method to check for fixing up paths.

View on GitHub (pinned to 9b90983fd2)

Solutions

1. Delete the corrupted segment from the local cache (or the whole cache dir) so it is re-downloaded.
2. Re-download/restore the segment from deep storage; verify checksums.
3. Re-ingest the affected data to rebuild the column.
4. Check disk health and deep-storage connectivity if corruption recurs.

// verify the column payload is present and plausibly sized before reading
java.io.File colFile = new File(segmentDir, columnName + "/" + versionFile);
if (!colFile.exists() || colFile.length() < MIN_HEADER_BYTES) {
  throw new CorruptSegmentException(columnName);
}

try {
  col = supplier.get();
} catch (RE e) {
  if (e.getMessage().startsWith("Failed to deserialize")) {
    log.error(e, "Corrupt/truncated nested column %s; evict cache entry and re-download", columnName);
    cacheEvictor.evict(segmentId);
  } else throw e;
}

When it happens

Trigger: Calling NestedDataColumnSupplier.get() on a column file whose payload is truncated, corrupted, or written in a form incompatible with the current reader despite a recognized version byte.

Common situations: Truncated segment files from failed deep-storage downloads or interrupted segment pushes; bit-rot on local segment cache disks; reading segments copied between clusters incompletely.

Understand the failure class

Background: "failed to read file", EACCES, ENOENT and "could not read <path>" errors: when a program can't read a file from disk — this error's family across 49 libraries.

Related errors

• Failed to deserialize V%s column.
• Failed to deserialize V%s column.
• Failed to deserialize V%s column.
• Input stream is null
• File[%s] too large[%d]

AI-assisted analysis of apache/druid@9b90983fd2 (2026-09-07). Data as JSON: /api/errors/a9b2190a6687abc2. Report an issue: GitHub.