{"record":{"id":"e7e6dcde3e10b5ba","repo":"pathwaycom/pathway","slug":"vector-column-name-r-has-type-dtype-which-is","errorCode":null,"errorMessage":"vector column {name!r} has type {dtype}, which is a multivector; a Pinecone record carries a single dense or sparse vector, so multivectors are not supported.","messagePattern":"vector column (.+?) has type (.+?), which is a multivector; a Pinecone record carries a single dense or sparse vector, so multivectors are not supported\\.","errorType":"validation","errorClass":"NotImplementedError","httpStatus":null,"severity":"error","filePath":"python/pathway/io/pinecone/__init__.py","lineNumber":85,"sourceCode":"\n    A dense vector is a numeric list / array, a sparse one a\n    ``list[tuple[int, float]]`` of ``(index, weight)`` pairs. Mirrors the runtime\n    ``PineconeError::InvalidVector`` / ``InvalidSparseVector`` guards so a wrong\n    column type fails at ``write()`` time rather than once data flows.\n    \"\"\"\n    if _is_statically_unknown(dtype):\n        return\n    if isinstance(dtype, dt.Optional):\n        raise ValueError(\n            f\"vector column {name!r} is nullable (type {dtype}); every row must \"\n            \"carry a vector, so the column cannot be optional.\"\n        )\n    if isinstance(dtype, dt.List):\n        inner = dtype.wrapped\n        if _is_numeric(inner) or _is_sparse_pair(inner):\n            return\n        if isinstance(inner, (dt.List, dt.Array)):\n            raise NotImplementedError(\n                f\"vector column {name!r} has type {dtype}, which is a multivector; \"\n                \"a Pinecone record carries a single dense or sparse vector, so \"\n                \"multivectors are not supported.\"\n            )\n    if isinstance(dtype, dt.Array) and _is_numeric(dtype.wrapped):\n        return\n    if isinstance(dtype, dt.Tuple) and all(_is_numeric(arg) for arg in dtype.args):\n        return\n    raise ValueError(\n        f\"vector column {name!r} has unsupported type {dtype}; a Pinecone vector \"\n        \"must be a list[float] or a 1-D float array (dense), or a \"\n        \"list[tuple[int, float]] of (index, weight) pairs (sparse).\"\n    )\n\n\ndef _check_metadata_dtype(name: str, dtype: dt.DType) -> None:\n    \"\"\"Reject a metadata column whose type Pinecone cannot store.\n","sourceCodeStart":67,"sourceCodeEnd":103,"githubUrl":"https://github.com/pathwaycom/pathway/blob/fa2f74a4649b7c5908690cf60137263d8d80de5f/python/pathway/io/pinecone/__init__.py#L67-L103","documentation":"A Pinecone record carries a single dense or sparse vector; a List-of-List (or List-of-Array) dtype is a multivector and cannot be stored. pw.io.pinecone.write raises this NotImplementedError at call time when the vector column's inner type is itself a list/array, so multivector attempts fail before the pipeline starts rather than per-row at the sink.","triggerScenarios":"Passing vector=table.vecs where table.vecs has dtype list[list[float]] — e.g. batching multiple embeddings per row — to pw.io.pinecone.write.","commonSituations":"Multi-chunk document pipelines that group several chunk embeddings into one row; switching from a multivector-capable store (e.g. some Milvus/Qdrant modes) to Pinecone; np.ndarray of shape (n_chunks, dim) serialized as nested lists.","solutions":["Flatten one-record-per-vector before the sink: explode/flatten the table so each row holds one embedding and a stable id (e.g. f\"{doc_id}-{i}\").","If only one vector per record is intended, fix the upstream step that wrapped embeddings in an extra list dimension.","Consider a store that supports multivectors if per-record multiple embeddings are a hard requirement."],"exampleFix":"# before\nclass Chunks(pw.Schema):\n    doc_id: str\n    embeddings: list[list[float]]  # multivector -> rejected\npw.io.pinecone.write(chunks, \"idx\", primary_key=chunks.doc_id, vector=chunks.embeddings)\n\n# after\nflat = chunks.flatten(pw.this.embeddings).with_columns(\n    vec_id=pw.this.doc_id + \"-\" + pw.this.meta.index.to_string()\n)\n# then write one embedding per row with vector=<the inner list column>","handlingStrategy":"validation","validationCode":"import pathway as pw\n\ndef is_multivector(dtype: pw.dt.DType) -> bool:\n    return (\n        isinstance(dtype, pw.dt.List)\n        and isinstance(dtype.wrapped, (pw.dt.List, pw.dt.Array))\n    )\n\nassert not is_multivector(table.schema[vector_col].dtype), (\n    \"Flatten to one embedding per row before Pinecone\"\n)","typeGuard":"import pathway as pw\n\ndef is_single_vector_dtype(dtype: pw.dt.DType) -> bool:\n    if isinstance(dtype, pw.dt.List):\n        inner = dtype.wrapped\n        return not isinstance(inner, (pw.dt.List, pw.dt.Array))\n    return True","tryCatchPattern":"try:\n    pw.io.pinecone.write(chunks, \"idx\", primary_key=chunks.doc_id, vector=chunks.embeddings)\nexcept NotImplementedError as e:\n    if \"multivector\" in str(e):\n        flat = chunks.flatten(pw.this.embeddings)\n        # assign one stable id per flattened row, then write\n    else:\n        raise","preventionTips":["Keep one embedding per row in tables destined for Pinecone.","For chunked documents, flatten chunks into separate rows with composite ids (doc_id-chunk_i).","Check the inner dtype of embedding columns after JSON/binary ingestion — extra nesting is common."],"tags":["pinecone","multivector","vector","dtype","pathway"],"backgroundTag":null,"analyzedSha":"fa2f74a4649b7c5908690cf60137263d8d80de5f","analyzedAt":"2026-08-15T01:48:17.006Z","schemaVersion":2},"datasetVersion":"2026-08-15T22:17:37.221Z"}