milvus-io/milvus · error · ValueError

Float16Vector size mismatch: expected {dim * 2}, got {len(by

Error message

Float16Vector size mismatch: expected {dim * 2}, got {len(bytes_data)}

What it means

Raised when a Float16Vector blob's byte count is not dim*2 (two bytes per half-precision component). Only fires when an explicit mismatching dim is passed or data is truncated. Caveat: this analyzer's float16 decoding is explicitly a 'simplified' placeholder (it reinterprets the raw half as uint16 and normalizes by 65535), so even sizes that pass validation decode to wrong values - do not trust the numeric output for float16 columns.

Source

Thrown at cmd/tools/binlogv2/parquet_analyzer/vector_deserializer.py:206

        Deserialize Float16Vector
        References Float16Vector processing logic from serde.go
        
        Args:
            bytes_data: byte data
            dim: dimension, if None will auto-calculate
            
        Returns:
            List[float]: deserialized float16 vector
        """
        if not bytes_data:
            return None
        
        try:
            if dim is None:
                dim = len(bytes_data) // 2
            
            if len(bytes_data) != dim * 2:
                raise ValueError(f"Float16Vector size mismatch: expected {dim * 2}, got {len(bytes_data)}")
            
            # Convert to float16 array
            float16_vector = []
            for i in range(0, len(bytes_data), 2):
                if i + 1 < len(bytes_data):
                    # Simple float16 conversion (simplified here)
                    uint16 = struct.unpack('<H', bytes_data[i:i+2])[0]
                    # Convert to float32 (simplified version)
                    float_val = float(uint16) / 65535.0  # normalization
                    float16_vector.append(float_val)
            
            return float16_vector
        
        except Exception as e:
            print(f"Float16Vector deserialization failed: {e}")
            return None
    
    @staticmethod

View on GitHub (pinned to b43a76673a)

Solutions

Omit dim so it is inferred (len//2) and cross-check with the schema dimension.
Verify the column truly is float16 (blob should be exactly half the byte count of a float32 vector of same dim).
For correct numeric values, decode with numpy: np.frombuffer(data, dtype='<f2').astype(np.float32) instead of the built-in simplified path.
Check truncation if blobs are systematically short.

Example fix

# before
vals = deserializer.deserialize_float16_vector(data, dim)

# after - correct IEEE 754 half decoding
import numpy as np
dim = len(data) // 2
assert len(data) == dim * 2
vals = np.frombuffer(data, dtype='<f2').astype(np.float32).tolist()

Defensive patterns

Strategy: validation

Validate before calling

def check_float16_vector(data: bytes, dim: int) -> bool:
    return len(data) > 0 and len(data) == dim * 2

Type guard

def is_valid_float16_blob(bytes_data: bytes, dim: int) -> bool:
    """True when bytes_data holds dim IEEE 754 half-precision values."""
    return isinstance(bytes_data, (bytes, bytearray)) and len(bytes_data) == dim * 2

Prevention

Validate len == dim*2 before decoding.
Do not trust the analyzer's simplified float16 math (uint16/65535) - use numpy '<f2' for real values.
Distinguish f16 (dim*2) from f32 (dim*4) blobs when the type label is uncertain.

When it happens

Trigger: Explicit dim disagreeing with blob length; truncated parquet data; passing float32 bytes (dim*4) while declaring the column Float16Vector.

Common situations: Schema migrated between float32 and float16; mislabeled vector type in the analyzer config; partial file export.

Related errors

AI-assisted analysis of milvus-io/milvus@b43a76673a (2026-08-15). Data as JSON: /api/errors/ab2eee22bce1ddea. Report an issue: GitHub.