{"record":{"id":"19666c50d817e52e","repo":"Mintplex-Labs/anything-llm","slug":"chromacloud-document-length-too-large-default-ma","errorCode":null,"errorMessage":"ChromaCloud::Document length too large (default max is ${this.limits.maxDocumentBytes}). Got ${testSubmission.document.length}. Upsert may fail!","messagePattern":"ChromaCloud::Document length too large \\(default max is (.+?)\\)\\. Got (.+?)\\. Upsert may fail!","errorType":"console","errorClass":null,"httpStatus":null,"severity":"warning","filePath":"server/utils/vectorDbProviders/chromacloud/index.js","lineNumber":71,"sourceCode":"   * @param {import(\"chromadb\").Collection} collection\n   * @param {{ids: string[], embeddings: number[], metadatas: Record<string, any>[], documents: string[]}[]} submissions\n   * @returns {Promise<boolean>} True if the upsert was successful, false otherwise.\n   * If the upsert was not successful, the error message will be returned.\n   */\n  async smartAdd(collection, submission) {\n    const testSubmission = {\n      id: submission.ids[0],\n      embedding: submission.embeddings[0],\n      metadata: submission.metadatas[0],\n      document: submission.documents[0],\n    };\n\n    if (testSubmission.embedding.length > this.limits.maxEmbeddingDim)\n      console.warn(\n        `ChromaCloud::Embedding dimension too large (default max is ${this.limits.maxEmbeddingDim}). Got ${testSubmission.embedding.length}. Upsert may fail!`\n      );\n    if (testSubmission.document.length > this.limits.maxDocumentBytes)\n      console.warn(\n        `ChromaCloud::Document length too large (default max is ${this.limits.maxDocumentBytes}). Got ${testSubmission.document.length}. Upsert may fail!`\n      );\n    if (\n      JSON.stringify(testSubmission.metadata).length >\n      this.limits.maxMetadataBytes\n    )\n      console.warn(\n        `ChromaCloud::Metadata length too large (default max is ${this.limits.maxMetadataBytes}). Got ${JSON.stringify(testSubmission.metadata).length}. Upsert may fail!`\n      );\n\n    // If the submissions are not too large, just add them directly.\n    if (submission.ids.length <= this.limits.maxRecordsPerWrite) {\n      await collection.add(submission);\n      return true;\n    }\n\n    this.logger(\n      `Upsert Payload is too large (max is ${this.limits.maxRecordsPerWrite} records). Splitting into chunks of ${this.limits.maxRecordsPerWrite} records.`","sourceCodeStart":53,"sourceCodeEnd":89,"githubUrl":"https://github.com/Mintplex-Labs/anything-llm/blob/3aec848f2885144aa8f1e53b9731a04310d5d558/server/utils/vectorDbProviders/chromacloud/index.js#L53-L89","documentation":"In ChromaCloud.smartAdd(), before upserting, the first record's document text (documents[0]) is compared against the maxDocumentBytes quota of 16384 (a JavaScript string length, i.e. characters, sampled per batch — not strictly bytes and not every record). When a chunk's text exceeds it, this warning prints and the cloud upsert will likely fail with 422. The provider itself chunks by maxRecordsPerWrite (300 records) but never splits oversized document text, so long chunks stay long.","triggerScenarios":"Embedding documents into a chromacloud workspace where a chunk's text exceeds 16,384 characters: a workspace text-splitter chunk size set very large, chunking effectively disabled for a pasted monolithic file, or HTML/scrape output whose cleaned text still yields huge chunks.","commonSituations":"Raising the workspace 'Text Chunk Size' preference far above defaults (e.g. 20000+) to 'keep more context', uploading a single giant text/PDF without splitting, migrating large-chunk workspaces from local Chroma (no byte limit) to Chroma Cloud, and non-ASCII content where characters≈bytes make the 16 KB budget tight.","solutions":["Lower the workspace text-splitter chunk size (and overlap) so each chunk stays comfortably under 16,384 characters — e.g. chunk size 1000–4000 — then re-embed the document.","If long chunks are intentional, use self-hosted Chroma (VECTOR_DB=chroma) or another vector DB without the per-document byte quota.","Re-upload/re-embed the affected documents after changing chunk settings; existing oversized chunks are not auto-fixed.","Watch the embed job logs: one oversized chunk warns but the rest of the batch continues — verify final document counts in the workspace."],"exampleFix":"# before: workspace chunk size large enough to exceed the Chroma Cloud quota\nworkspace.textSplitterChunkSize = 20000  # chunk > 16,384 chars → \"Document length too large\"\n\n# after\nworkspace.textSplitterChunkSize = 2000    # every chunk well under 16,384 chars","handlingStrategy":"validation","validationCode":"const CHROMA_CLOUD_MAX_DOC_CHARS = 16_384;\n\nconst chunks = toChunks(text, workspace.chunkSize).map(chunk => {\n  if (chunk.text && chunk.text.length > CHROMA_CLOUD_MAX_DOC_CHARS) {\n    throw new Error(\n      `Chunk of ${chunk.text.length} chars exceeds Chroma Cloud's 16,384-char document limit — lower the workspace chunk size.`\n    );\n  }\n  return chunk;\n});","typeGuard":null,"tryCatchPattern":null,"preventionTips":["Keep workspace text-splitter chunk size in the 1000–4000 range; big chunks rarely help retrieval but reliably trip cloud quotas.","Validate chunk lengths in the ingestion pipeline before upsert, not after the provider warns.","If you rely on very large chunks, that is a signal to self-host Chroma (no per-record byte limits) rather than fight the quota."],"tags":["chroma-cloud","chunking","vector-db","quotas","payload-size"],"backgroundTag":"payload-too-large","analyzedSha":"3aec848f2885144aa8f1e53b9731a04310d5d558","analyzedAt":"2026-08-18T10:02:21.017Z","contentChangedAt":"2026-08-18T10:02:21.017Z","schemaVersion":2},"datasetVersion":"2026-09-08T15:18:49.778Z"}