{"record":{"id":"bb117ee32905286d","repo":"pathwaycom/pathway","slug":"milvus-collection-collection-name-r-does-not-exi","errorCode":null,"errorMessage":"Milvus collection {collection_name!r} does not exist; create it before writing. pw.io.milvus.write never creates a collection because it cannot infer the vector field's dimension.","messagePattern":"Milvus collection (.+?) does not exist; create it before writing\\. pw\\.io\\.milvus\\.write never creates a collection because it cannot infer the vector field's dimension\\.","errorType":"validation","errorClass":"ValueError","httpStatus":null,"severity":"error","filePath":"python/pathway/io/milvus/__init__.py","lineNumber":305,"sourceCode":"\n    if batch_size < 1:\n        raise ValueError(f\"batch_size must be a positive integer, got {batch_size}.\")\n\n    if primary_key._table is not table:\n        raise ValueError(\n            f\"primary_key column {primary_key._name!r} does not belong to the \"\n            f\"provided table. Pass a column reference from the same table, \"\n            f\"e.g. primary_key=table.{primary_key._name}.\"\n        )\n\n    client = _make_client(MilvusClient, uri)\n\n    # Fail fast if the collection is missing: otherwise the error would only\n    # surface deep inside pw.run() on the first upsert, or — for an empty table —\n    # never, silently running a misconfigured pipeline that writes nothing.\n    if not client.has_collection(collection_name):\n        client.close()\n        raise ValueError(\n            f\"Milvus collection {collection_name!r} does not exist; create it \"\n            f\"before writing. pw.io.milvus.write never creates a collection \"\n            f\"because it cannot infer the vector field's dimension.\"\n        )\n\n    pk = primary_key._name\n    # Accumulates (is_addition, row) in arrival order for the current batch.\n    _buffer: list[tuple] = []\n\n    def on_change(key, row, time, is_addition):\n        _buffer.append((is_addition, _prepare_row(row)))\n\n    def on_time_end(time):\n        to_delete = []\n        to_upsert = []\n\n        for is_add, row in _buffer:\n            if is_add:","sourceCodeStart":287,"sourceCodeEnd":323,"githubUrl":"https://github.com/pathwaycom/pathway/blob/fa2f74a4649b7c5908690cf60137263d8d80de5f/python/pathway/io/milvus/__init__.py#L287-L323","documentation":"pw.io.milvus.write never creates collections, because it cannot infer the vector field's dimension from a Pathway table alone. Before writing, it checks client.has_collection(collection_name) and, if the collection is missing, closes the client and raises ValueError telling you to create it first. This fails fast instead of erroring deep inside pw.run() — or, for an empty table, never at all.","triggerScenarios":"Calling pw.io.milvus.write(table, uri, collection_name='docs', ...) before any collection named 'docs' exists on the server; also after dropping the collection or pointing collection_name at a typo'd name.","commonSituations":"First run of a new pipeline against a fresh Milvus instance; typos in collection_name; environment mismatch (writing to a dev URI while the collection was created in another environment).","solutions":["Create the collection up front with pymilvus MilvusClient.create_schema/create_collection, defining the vector field with the correct dimension matching your embeddings","Verify the name: print(client.list_collections()) and fix the collection_name typo","Keep a small bootstrap script that ensures the schema exists and run it before pipeline startup / deployment"],"exampleFix":"# before\npw.io.milvus.write(t, uri=\"./milvus.db\", collection_name=\"docs\", primary_key=t.id)  # 'docs' missing\n\n# after\nfrom pymilvus import MilvusClient, DataType\nclient = MilvusClient(\"./milvus.db\")\nif not client.has_collection(\"docs\"):\n    schema = client.create_schema(auto_id=False)\n    schema.add_field(\"id\", DataType.INT64, is_primary=True)\n    schema.add_field(\"vector\", DataType.FLOAT_VECTOR, dim=384)\n    client.create_collection(\"docs\", schema)\npw.io.milvus.write(t, uri=\"./milvus.db\", collection_name=\"docs\", primary_key=t.id)","handlingStrategy":"validation","validationCode":"from pymilvus import MilvusClient\n\nclient = MilvusClient(uri)\nif not client.has_collection(collection_name):\n    raise ValueError(f\"Create collection {collection_name!r} (with vector dim) before pw.io.milvus.write\")\nclient.close()","typeGuard":null,"tryCatchPattern":"try:\n    pw.io.milvus.write(table, uri, collection_name, primary_key=table.id)\nexcept ValueError as e:\n    if \"does not exist\" in str(e):\n        ensure_collection(uri, collection_name, dim=len(table.emb[0]))  # your bootstrap\n    else:\n        raise","preventionTips":["Run a bootstrap script that creates the collection (with explicit vector dimension) before the pipeline","Check client.list_collections() when writes target a shared server","Drive collection_name from one shared constant to avoid environment drift"],"tags":["milvus","pathway","collection","schema","setup"],"backgroundTag":null,"analyzedSha":"fa2f74a4649b7c5908690cf60137263d8d80de5f","analyzedAt":"2026-08-15T01:48:17.006Z","schemaVersion":2},"datasetVersion":"2026-08-15T22:17:37.221Z"}