{"record":{"id":"634a39266f020669","repo":"FlowiseAI/Flowise","slug":"the-field-field-is-not-provided-in-documents","errorCode":null,"errorMessage":"The field \"${field}\" is not provided in documents[${index}].metadata.","messagePattern":"The field \"(.+?)\" is not provided in documents\\[(.+?)\\]\\.metadata\\.","errorType":"validation","errorClass":"Error","httpStatus":null,"severity":"error","filePath":"packages/components/nodes/vectorstores/Milvus/Milvus.ts","lineNumber":451,"sourceCode":"                    case this.primaryField:\n                        if (!this.autoId) {\n                            if (doc.metadata[this.primaryField] === undefined) {\n                                throw new Error(\n                                    `The Collection's primaryField is configured with autoId=false, thus its value must be provided through metadata.`\n                                )\n                            }\n                            data[field] = doc.metadata[this.primaryField]\n                        }\n                        break\n                    case this.textField:\n                        data[field] = doc.pageContent\n                        break\n                    case this.vectorField:\n                        data[field] = vec\n                        break\n                    default: // metadata fields\n                        if (doc.metadata[field] === undefined) {\n                            throw new Error(`The field \"${field}\" is not provided in documents[${index}].metadata.`)\n                        } else if (typeof doc.metadata[field] === 'object') {\n                            data[field] = JSON.stringify(doc.metadata[field])\n                        } else {\n                            data[field] = doc.metadata[field]\n                        }\n                        break\n                }\n            })\n\n            insertDatas.push(data)\n        }\n\n        const descIndexResp = await this.client.describeIndex({\n            collection_name: this.collectionName\n        })\n\n        if (descIndexResp.status.error_code === ErrorCode.IndexNotExist) {\n            const resp = await this.client.createIndex({","sourceCodeStart":433,"sourceCodeEnd":469,"githubUrl":"https://github.com/FlowiseAI/Flowise/blob/abe4a8601a058047b350c260676826e21dd14101/packages/components/nodes/vectorstores/Milvus/Milvus.ts#L433-L469","documentation":"Thrown during Milvus upsert when iterating the collection's declared fields and encountering a non-primary/text/vector field whose value is missing from the current document's `metadata`. Milvus requires every schema field to have a value on insert, so any declared metadata column without a corresponding `doc.metadata[field]` entry aborts the batch.","triggerScenarios":"The Milvus collection schema declares extra scalar columns (e.g. `source`, `category`, `timestamp`) but one or more ingested `Document` objects omit that key in `metadata`. The check is `doc.metadata[field] === undefined`, so `null` also trips a later path.","commonSituations":"Schema evolved (new column added) but upstream documents were not updated; documents come from heterogeneous loaders where some set the field and others do not; field name casing mismatch between schema and metadata.","solutions":["Ensure every document's `metadata` contains a value for every non-primary/text/vector field declared in the collection schema.","Backfill missing fields with a sensible default (e.g. empty string) before insert.","Drop the extra scalar field from the Milvus schema if it is not consistently populated.","Log `this.fields` vs the document's metadata keys before insert to surface gaps early."],"exampleFix":"// before — gaps in metadata cause per-document abort\nfor (const doc of documents) await store.addDocuments([doc])\n// after — backfill all declared fields with defaults\nconst requiredFields = store.fields.filter(f => ![store.primaryField, store.textField, store.vectorField].includes(f))\nconst safeDocs = documents.map(doc => {\n    const md = { ...doc.metadata }\n    for (const f of requiredFields) if (md[f] === undefined) md[f] = ''\n    return new Document({ pageContent: doc.pageContent, metadata: md })\n})","handlingStrategy":"validation","validationCode":"function validateMetadataFields(docs: Document[], requiredFields: string[], reserved: string[]) {\n  const need = requiredFields.filter(f => !reserved.includes(f))\n  for (let i = 0; i < docs.length; i++) {\n    for (const f of need) {\n      if (docs[i].metadata?.[f] === undefined) {\n        throw new Error(`documents[${i}].metadata missing field '${f}'`)\n      }\n    }\n  }\n}","typeGuard":"function documentCoversSchema(doc: Document, schemaFields: string[]): boolean {\n  return schemaFields.every(f => doc.metadata?.[f] !== undefined)\n}","tryCatchPattern":"try {\n  validateMetadataFields(documents, this.fields, [this.primaryField, this.textField, this.vectorField])\n  await store.addVectors(vectors, documents)\n} catch (e) {\n  throw e instanceof Error ? e : new Error(String(e))\n}","preventionTips":["Keep document metadata in sync with the collection schema; backfill new scalar columns with defaults.","Validate every document covers all declared schema fields before batch insert.","Use a single loader pipeline that guarantees required metadata keys.","Log field-level gaps before they reach Milvus."],"tags":["milvus","upsert","validation","metadata","schema"],"backgroundTag":null,"analyzedSha":"abe4a8601a058047b350c260676826e21dd14101","analyzedAt":"2026-08-12T16:04:40.823Z","schemaVersion":2},"datasetVersion":"2026-08-12T18:17:37.767Z"}