{"record":{"id":"fec60288f9865cc8","repo":"chroma-core/chroma","slug":"record-set-length-recordsetlength-exceeds-max-b","errorCode":null,"errorMessage":"Record set length ${recordSetLength} exceeds max batch size ${maxBatchSize}","messagePattern":"Record set length (.+?) exceeds max batch size (.+?)","errorType":"validation","errorClass":"ChromaValueError","httpStatus":null,"severity":"error","filePath":"clients/new-js/packages/chromadb/src/utils.ts","lineNumber":522,"sourceCode":"\n  if (recordSet.documents) {\n    validateDocuments({\n      documents: recordSet.documents,\n      fieldName: documentsField,\n    });\n  }\n\n  if (recordSet.metadatas) {\n    validateMetadatas(recordSet.metadatas);\n  }\n};\n\nexport const validateMaxBatchSize = (\n  recordSetLength: number,\n  maxBatchSize: number,\n) => {\n  if (recordSetLength > maxBatchSize) {\n    throw new ChromaValueError(\n      `Record set length ${recordSetLength} exceeds max batch size ${maxBatchSize}`,\n    );\n  }\n};\n\n/**\n * Validates a where clause for metadata filtering.\n * @param where - Where clause object to validate\n * @throws ChromaValueError if the where clause is malformed\n */\nexport const validateWhere = (where: Where) => {\n  if (typeof where !== \"object\") {\n    throw new ChromaValueError(\"Expected where to be a non-empty object\");\n  }\n\n  if (Object.keys(where).length != 1) {\n    throw new ChromaValueError(\n      `Expected 'where' to have exactly one operator, but got ${","sourceCodeStart":504,"sourceCodeEnd":540,"githubUrl":"https://github.com/chroma-core/chroma/blob/aecdd12c8a891610db8653630b066b32ceb678b5/clients/new-js/packages/chromadb/src/utils.ts#L504-L540","documentation":"validateMaxBatchSize throws when the number of records in one add/upsert call exceeds the server's advertised maximum batch size (obtained during collection pre-flight). The client enforces the server limit locally so oversized batches fail fast instead of producing a server-side 4xx. The limit comes from the Chroma server configuration (max batch size), not from the client.","triggerScenarios":"One collection.add() with e.g. 100k records against a server whose max batch size is lower. Bulk import scripts that push a whole dataset in a single call without chunking.","commonSituations":"Initial bulk ingestion from a CSV/DataFrame; migrating a large corpus into a new collection; server upgraded or reconfigured with a smaller batch limit than the client assumed.","solutions":["Chunk the payload: loop add() over slices of a fixed size (e.g. 5k) below the server limit.","Raise the max batch size in the Chroma server configuration if your deployment allows it, then restart the server.","Stream records in fixed-size chunks so you never depend on the exact server limit."],"exampleFix":"// before\nawait collection.add({ ids, documents }); // 100k records in one call\n\n// after\nconst CHUNK = 5000;\nfor (let i = 0; i < ids.length; i += CHUNK) {\n  await collection.add({ ids: ids.slice(i, i + CHUNK), documents: documents.slice(i, i + CHUNK) });\n}","handlingStrategy":"validation","validationCode":"const CHUNK = 5000; // keep well under the server max batch size\nfor (let i = 0; i < records.ids.length; i += CHUNK) {\n  await collection.add({\n    ids: records.ids.slice(i, i + CHUNK),\n    documents: records.documents?.slice(i, i + CHUNK),\n  });\n}","typeGuard":null,"tryCatchPattern":"try {\n  await collection.add(batch);\n} catch (e) {\n  if ((e as Error).message.includes('exceeds max batch size')) {\n    // split the batch in half and recurse until under the limit\n  } else {\n    throw e;\n  }\n}","preventionTips":["Chunk every bulk import; never send an unbounded batch.","Record the server's max batch size at startup and size chunks from it.","Raise max batch size in server config only after measuring memory impact."],"tags":["batching","limits","bulk-import","validation"],"backgroundTag":"batch-size-exceeded","analyzedSha":"aecdd12c8a891610db8653630b066b32ceb678b5","analyzedAt":"2026-08-16T21:53:27.228Z","schemaVersion":2},"datasetVersion":"2026-08-16T23:17:17.608Z"}