{"record":{"id":"a87b4acb0239dc23","repo":"milvus-io/milvus","slug":"int8vector-size-mismatch-expected-dim-got-len","errorCode":null,"errorMessage":"Int8Vector size mismatch: expected {dim}, got {len(bytes_data)}","messagePattern":"Int8Vector size mismatch: expected (.+?), got (.+?)","errorType":"console","errorClass":"ValueError","httpStatus":null,"severity":"error","filePath":"cmd/tools/binlogv2/parquet_analyzer/vector_deserializer.py","lineNumber":175,"sourceCode":"        Deserialize Int8Vector\n        References Int8Vector processing logic from serde.go\n        \n        Args:\n            bytes_data: byte data\n            dim: dimension, if None will auto-calculate\n            \n        Returns:\n            List[int]: deserialized int8 vector\n        \"\"\"\n        if not bytes_data:\n            return None\n        \n        try:\n            if dim is None:\n                dim = len(bytes_data)\n            \n            if len(bytes_data) != dim:\n                raise ValueError(f\"Int8Vector size mismatch: expected {dim}, got {len(bytes_data)}\")\n            \n            # Convert to int8 array\n            int8_vector = [int8 for int8 in bytes_data]\n            return int8_vector\n        \n        except Exception as e:\n            print(f\"Int8Vector deserialization failed: {e}\")\n            return None\n    \n    @staticmethod\n    def deserialize_float16_vector(bytes_data: bytes, dim: Optional[int] = None) -> Optional[List[float]]:\n        \"\"\"\n        Deserialize Float16Vector\n        References Float16Vector processing logic from serde.go\n        \n        Args:\n            bytes_data: byte data\n            dim: dimension, if None will auto-calculate","sourceCodeStart":157,"sourceCodeEnd":193,"githubUrl":"https://github.com/milvus-io/milvus/blob/b43a76673a9fe5f01f158979731dc8fd542df81f/cmd/tools/binlogv2/parquet_analyzer/vector_deserializer.py#L157-L193","documentation":"Raised when an Int8Vector blob's byte count differs from the expected dim. Int8 vectors store exactly one signed byte per dimension, so len(bytes) must equal dim exactly. Because the deserializer infers dim = len(bytes_data) when dim is None, this only triggers when an explicit mismatching dim is passed or the blob is truncated.","triggerScenarios":"Explicit dim supplied that differs from blob length; truncated parquet data; column actually another vector type being read as Int8Vector.","commonSituations":"Dimension changed in schema between write and analysis; CLI dim flag typo; corrupt/truncated export.","solutions":["Omit dim (or pass dim=len(data)) and validate the inferred dimension against the collection schema.","Verify file integrity and re-export if blobs are short.","Confirm the column type mapping labels the field Int8Vector correctly."],"exampleFix":"# before\ndeserializer.deserialize_int8_vector(data, dim=128)  # data len 127\n\n# after\ndim = len(data)\ndeserializer.deserialize_int8_vector(data, dim)","handlingStrategy":"validation","validationCode":"def check_int8_vector(data: bytes, dim: int) -> bool:\n    return len(data) > 0 and len(data) == dim","typeGuard":"def is_valid_int8_vector_blob(bytes_data: bytes, dim: int) -> bool:\n    \"\"\"True when bytes_data has exactly one byte per dimension.\"\"\"\n    return isinstance(bytes_data, (bytes, bytearray)) and len(bytes_data) == dim","tryCatchPattern":null,"preventionTips":["Int8Vector is 1 byte per dim - simplest invariant, check len(data) == dim first.","Let dim be inferred when unsure; cross-check with the schema afterwards.","Check stdout for the swallowed 'Int8Vector deserialization failed' message."],"tags":["python","binlog","parquet","vector","data-validation"],"backgroundTag":null,"analyzedSha":"b43a76673a9fe5f01f158979731dc8fd542df81f","analyzedAt":"2026-08-15T10:29:28.408Z","schemaVersion":2},"datasetVersion":"2026-08-15T22:17:37.221Z"}