milvus-io/milvus · error · ValueError

FloatVector size mismatch: expected {dim * 4}, got {len(byte

Error message

FloatVector size mismatch: expected {dim * 4}, got {len(bytes_data)}

What it means

Raised by the binlogv2 parquet analyzer when a FloatVector column's raw byte blob length is not exactly dim*4 bytes. Float vectors are stored as packed little-endian float32, so the analyzer can validate the byte count against the schema dimension; a mismatch means the parquet data is truncated, the dim passed in is wrong, or the column is not actually a float32 vector.

Source

Thrown at cmd/tools/binlogv2/parquet_analyzer/vector_deserializer.py:106

        Deserialize FloatVector
        References FloatVector processing logic from serde.go
        
        Args:
            bytes_data: byte data
            dim: dimension, if None will auto-calculate
            
        Returns:
            List[float]: deserialized float vector
        """
        if not bytes_data:
            return None
        
        try:
            if dim is None:
                dim = len(bytes_data) // 4
            
            if len(bytes_data) != dim * 4:
                raise ValueError(f"FloatVector size mismatch: expected {dim * 4}, got {len(bytes_data)}")
            
            # Use struct to unpack float32 data
            floats = struct.unpack(f'<{dim}f', bytes_data)
            return list(floats)
        
        except Exception as e:
            print(f"FloatVector deserialization failed: {e}")
            return None
    
    @staticmethod
    def deserialize_binary_vector(bytes_data: bytes, dim: Optional[int] = None) -> Optional[List[int]]:
        """
        Deserialize BinaryVector
        References BinaryVector processing logic from serde.go
        
        Args:
            bytes_data: byte data
            dim: dimension, if None will auto-calculate

View on GitHub (pinned to b43a76673a)

Solutions

1. Confirm the dimension from the collection schema and pass it explicitly so it matches the data written.
2. Check file integrity: re-export the binlog/parquet or verify file size; truncation is the most common cause of short blobs.
3. Verify the column really is FloatVector (not Float16/Binary) in the analyzer's type mapping.
4. Note the function prints the error and returns None - downstream cells will show empty/None, so search stdout for 'FloatVector deserialization failed' to find the offending rows.

Example fix

# before
dim = 128  # hardcoded
deserializer.deserialize_float_vector(data, dim)

# after - derive dim from schema before analysis
dim = collection_schema['vector_field'].dim
assert len(data) % 4 == 0, f'not float32-aligned: {len(data)} bytes'
deserializer.deserialize_float_vector(data, dim)

def check_float_vector(data: bytes, dim: int) -> bool:
    return len(data) > 0 and len(data) % 4 == 0 and len(data) == dim * 4

def is_valid_float_vector_blob(bytes_data: bytes, dim: int) -> bool:
    """True when bytes_data is a complete packed-float32 vector of dimension dim."""
    return isinstance(bytes_data, (bytes, bytearray)) and len(bytes_data) == dim * 4

When it happens

Trigger: Running the analyzer with a --dim that disagrees with the actual data; parquet file truncated mid-row-group; reading a column that stores a different vector type (e.g. float16 bytes) while labeling it FloatVector.

Common situations: Schema evolved (dimension changed) and old files are analyzed with the new dim; partial/corrupted binlog export; CLI flag dim omitted causing the wrong inference path.

Related errors

• BinaryVector size mismatch: expected {expected_size}, got {l
• Int8Vector size mismatch: expected {dim}, got {len(bytes_dat
• Float16Vector size mismatch: expected {dim * 2}, got {len(by
• BFloat16Vector size mismatch: expected {dim * 2}, got {len(b
• Empty payload received

AI-assisted analysis of milvus-io/milvus@b43a76673a (2026-08-15). Data as JSON: /api/errors/d7a54d6a865aded5. Report an issue: GitHub.