milvus-io/milvus · error · ValueError

BinaryVector size mismatch: expected

Error message

BinaryVector size mismatch: expected {expected_size}, got {len(bytes_data)}

What it means

Raised when a BinaryVector blob's byte length does not equal ceil(dim/8). Binary vectors pack dim bits, 8 per byte, MSB-first per Milvus storage; the analyzer validates the byte count against the schema dimension and rejects mismatches. Note the deserializer itself infers dim = len*8 when dim is None, so this fires only when an explicit wrong dim is supplied or the blob is truncated.

Solutions

1. Omit the explicit dim argument and let it be inferred from the byte length, then cross-check against the schema.
2. Verify dim from collection schema and confirm ceil(dim/8) equals the blob length.
3. Check the parquet file for truncation if lengths are systematically short.

Example fix

# before
deserializer.deserialize_binary_vector(data, dim=64)  # blob is 16 bytes

# after - let dim be inferred, or validate first
dim = len(data) * 8
expected = (dim + 7) // 8
assert len(data) == expected
deserializer.deserialize_binary_vector(data, dim)

import math
def check_binary_vector(data: bytes, dim: int) -> bool:
    return len(data) > 0 and len(data) == math.ceil(dim / 8)

def is_valid_binary_vector_blob(bytes_data: bytes, dim: int) -> bool:
    """True when bytes_data holds exactly dim bits, 8 per byte."""
    import math
    return isinstance(bytes_data, (bytes, bytearray)) and len(bytes_data) == math.ceil(dim / 8)

When it happens

Trigger: Passing an explicit dim that disagrees with the blob length (e.g. dim=100 for 13 bytes, since ceil(100/8)=13 matches only specific values); truncated parquet data; column mislabeled as BinaryVector.

Common situations: Schema dim changed after files were written; manual dim entry on the command line; reading mixed-type vector columns with one blanket type.

Related errors

• BFloat16Vector size mismatch: expected
• Float16Vector size mismatch: expected
• FloatVector size mismatch: expected
• Int8Vector size mismatch: expected
• Empty payload received

AI-assisted analysis of milvus-io/milvus@b43a76673a (2026-08-15). Data as JSON: /api/errors/726d3fae8c215046. Report an issue: GitHub.

Appendix: source

Thrown at cmd/tools/binlogv2/parquet_analyzer/vector_deserializer.py:138

        References BinaryVector processing logic from serde.go
        
        Args:
            bytes_data: byte data
            dim: dimension, if None will auto-calculate
            
        Returns:
            List[int]: deserialized binary vector
        """
        if not bytes_data:
            return None
        
        try:
            if dim is None:
                dim = len(bytes_data) * 8
            
            expected_size = (dim + 7) // 8
            if len(bytes_data) != expected_size:
                raise ValueError(f"BinaryVector size mismatch: expected {expected_size}, got {len(bytes_data)}")
            
            # Convert to binary vector
            binary_vector = []
            for byte in bytes_data:
                for i in range(8):
                    bit = (byte >> i) & 1
                    binary_vector.append(bit)
            
            # Only return first dim elements
            return binary_vector[:dim]
        
        except Exception as e:
            print(f"BinaryVector deserialization failed: {e}")
            return None
    
    @staticmethod
    def deserialize_int8_vector(bytes_data: bytes, dim: Optional[int] = None) -> Optional[List[int]]:
        """

View on GitHub (pinned to b43a76673a)