{"record":{"id":"d7a54d6a865aded5","repo":"milvus-io/milvus","slug":"floatvector-size-mismatch-expected-dim-4-got","errorCode":null,"errorMessage":"FloatVector size mismatch: expected {dim * 4}, got {len(bytes_data)}","messagePattern":"FloatVector size mismatch: expected (.+?), got (.+?)","errorType":"console","errorClass":"ValueError","httpStatus":null,"severity":"error","filePath":"cmd/tools/binlogv2/parquet_analyzer/vector_deserializer.py","lineNumber":106,"sourceCode":"        Deserialize FloatVector\n        References FloatVector processing logic from serde.go\n        \n        Args:\n            bytes_data: byte data\n            dim: dimension, if None will auto-calculate\n            \n        Returns:\n            List[float]: deserialized float vector\n        \"\"\"\n        if not bytes_data:\n            return None\n        \n        try:\n            if dim is None:\n                dim = len(bytes_data) // 4\n            \n            if len(bytes_data) != dim * 4:\n                raise ValueError(f\"FloatVector size mismatch: expected {dim * 4}, got {len(bytes_data)}\")\n            \n            # Use struct to unpack float32 data\n            floats = struct.unpack(f'<{dim}f', bytes_data)\n            return list(floats)\n        \n        except Exception as e:\n            print(f\"FloatVector deserialization failed: {e}\")\n            return None\n    \n    @staticmethod\n    def deserialize_binary_vector(bytes_data: bytes, dim: Optional[int] = None) -> Optional[List[int]]:\n        \"\"\"\n        Deserialize BinaryVector\n        References BinaryVector processing logic from serde.go\n        \n        Args:\n            bytes_data: byte data\n            dim: dimension, if None will auto-calculate","sourceCodeStart":88,"sourceCodeEnd":124,"githubUrl":"https://github.com/milvus-io/milvus/blob/b43a76673a9fe5f01f158979731dc8fd542df81f/cmd/tools/binlogv2/parquet_analyzer/vector_deserializer.py#L88-L124","documentation":"Raised by the binlogv2 parquet analyzer when a FloatVector column's raw byte blob length is not exactly dim*4 bytes. Float vectors are stored as packed little-endian float32, so the analyzer can validate the byte count against the schema dimension; a mismatch means the parquet data is truncated, the dim passed in is wrong, or the column is not actually a float32 vector.","triggerScenarios":"Running the analyzer with a --dim that disagrees with the actual data; parquet file truncated mid-row-group; reading a column that stores a different vector type (e.g. float16 bytes) while labeling it FloatVector.","commonSituations":"Schema evolved (dimension changed) and old files are analyzed with the new dim; partial/corrupted binlog export; CLI flag dim omitted causing the wrong inference path.","solutions":["Confirm the dimension from the collection schema and pass it explicitly so it matches the data written.","Check file integrity: re-export the binlog/parquet or verify file size; truncation is the most common cause of short blobs.","Verify the column really is FloatVector (not Float16/Binary) in the analyzer's type mapping.","Note the function prints the error and returns None - downstream cells will show empty/None, so search stdout for 'FloatVector deserialization failed' to find the offending rows."],"exampleFix":"# before\ndim = 128  # hardcoded\ndeserializer.deserialize_float_vector(data, dim)\n\n# after - derive dim from schema before analysis\ndim = collection_schema['vector_field'].dim\nassert len(data) % 4 == 0, f'not float32-aligned: {len(data)} bytes'\ndeserializer.deserialize_float_vector(data, dim)","handlingStrategy":"validation","validationCode":"def check_float_vector(data: bytes, dim: int) -> bool:\n    return len(data) > 0 and len(data) % 4 == 0 and len(data) == dim * 4","typeGuard":"def is_valid_float_vector_blob(bytes_data: bytes, dim: int) -> bool:\n    \"\"\"True when bytes_data is a complete packed-float32 vector of dimension dim.\"\"\"\n    return isinstance(bytes_data, (bytes, bytearray)) and len(bytes_data) == dim * 4","tryCatchPattern":null,"preventionTips":["Always pass dim from the collection schema rather than hardcoding.","Sanity-check len(data) % 4 == 0 before attempting deserialization.","Watch stdout for 'FloatVector deserialization failed' - the helper swallows the exception and returns None."],"tags":["python","binlog","parquet","vector","data-validation"],"backgroundTag":null,"analyzedSha":"b43a76673a9fe5f01f158979731dc8fd542df81f","analyzedAt":"2026-08-15T10:29:28.408Z","schemaVersion":2},"datasetVersion":"2026-08-15T17:31:12.345Z"}