milvus-io/milvus · error · ValueError

BinaryVector size mismatch: expected {expected_size}, got {l

Error message

BinaryVector size mismatch: expected {expected_size}, got {len(bytes_data)}

What it means

Raised when a BinaryVector blob's byte length does not equal ceil(dim/8). Binary vectors pack dim bits, 8 per byte, MSB-first per Milvus storage; the analyzer validates the byte count against the schema dimension and rejects mismatches. Note the deserializer itself infers dim = len*8 when dim is None, so this fires only when an explicit wrong dim is supplied or the blob is truncated.

Source

Thrown at cmd/tools/binlogv2/parquet_analyzer/vector_deserializer.py:138

        References BinaryVector processing logic from serde.go
        
        Args:
            bytes_data: byte data
            dim: dimension, if None will auto-calculate
            
        Returns:
            List[int]: deserialized binary vector
        """
        if not bytes_data:
            return None
        
        try:
            if dim is None:
                dim = len(bytes_data) * 8
            
            expected_size = (dim + 7) // 8
            if len(bytes_data) != expected_size:
                raise ValueError(f"BinaryVector size mismatch: expected {expected_size}, got {len(bytes_data)}")
            
            # Convert to binary vector
            binary_vector = []
            for byte in bytes_data:
                for i in range(8):
                    bit = (byte >> i) & 1
                    binary_vector.append(bit)
            
            # Only return first dim elements
            return binary_vector[:dim]
        
        except Exception as e:
            print(f"BinaryVector deserialization failed: {e}")
            return None
    
    @staticmethod
    def deserialize_int8_vector(bytes_data: bytes, dim: Optional[int] = None) -> Optional[List[int]]:
        """

View on GitHub (pinned to b43a76673a)

Solutions

Omit the explicit dim argument and let it be inferred from the byte length, then cross-check against the schema.
Verify dim from collection schema and confirm ceil(dim/8) equals the blob length.
Check the parquet file for truncation if lengths are systematically short.

Example fix

# before
deserializer.deserialize_binary_vector(data, dim=64)  # blob is 16 bytes

# after - let dim be inferred, or validate first
dim = len(data) * 8
expected = (dim + 7) // 8
assert len(data) == expected
deserializer.deserialize_binary_vector(data, dim)

Defensive patterns

Strategy: validation

Validate before calling

import math
def check_binary_vector(data: bytes, dim: int) -> bool:
    return len(data) > 0 and len(data) == math.ceil(dim / 8)

Type guard

def is_valid_binary_vector_blob(bytes_data: bytes, dim: int) -> bool:
    """True when bytes_data holds exactly dim bits, 8 per byte."""
    import math
    return isinstance(bytes_data, (bytes, bytearray)) and len(bytes_data) == math.ceil(dim / 8)

Prevention

Prefer omitting dim so it is inferred from blob length, then compare against the schema.
Remember binary vectors are bit-packed: expected bytes = ceil(dim/8), not dim.
Verify parquet exports completed fully; truncation is the top cause of size mismatches.

When it happens

Trigger: Passing an explicit dim that disagrees with the blob length (e.g. dim=100 for 13 bytes, since ceil(100/8)=13 matches only specific values); truncated parquet data; column mislabeled as BinaryVector.

Common situations: Schema dim changed after files were written; manual dim entry on the command line; reading mixed-type vector columns with one blanket type.

Related errors

AI-assisted analysis of milvus-io/milvus@b43a76673a (2026-08-15). Data as JSON: /api/errors/726d3fae8c215046. Report an issue: GitHub.