Mintplex-Labs/anything-llm · warning

ChromaCloud::Document length too large (default max is ${thi

Error message

ChromaCloud::Document length too large (default max is ${this.limits.maxDocumentBytes}). Got ${testSubmission.document.length}. Upsert may fail!

What it means

In ChromaCloud.smartAdd(), before upserting, the first record's document text (documents[0]) is compared against the maxDocumentBytes quota of 16384 (a JavaScript string length, i.e. characters, sampled per batch — not strictly bytes and not every record). When a chunk's text exceeds it, this warning prints and the cloud upsert will likely fail with 422. The provider itself chunks by maxRecordsPerWrite (300 records) but never splits oversized document text, so long chunks stay long.

Source

Thrown at server/utils/vectorDbProviders/chromacloud/index.js:71

   * @param {import("chromadb").Collection} collection
   * @param {{ids: string[], embeddings: number[], metadatas: Record<string, any>[], documents: string[]}[]} submissions
   * @returns {Promise<boolean>} True if the upsert was successful, false otherwise.
   * If the upsert was not successful, the error message will be returned.
   */
  async smartAdd(collection, submission) {
    const testSubmission = {
      id: submission.ids[0],
      embedding: submission.embeddings[0],
      metadata: submission.metadatas[0],
      document: submission.documents[0],
    };

    if (testSubmission.embedding.length > this.limits.maxEmbeddingDim)
      console.warn(
        `ChromaCloud::Embedding dimension too large (default max is ${this.limits.maxEmbeddingDim}). Got ${testSubmission.embedding.length}. Upsert may fail!`
      );
    if (testSubmission.document.length > this.limits.maxDocumentBytes)
      console.warn(
        `ChromaCloud::Document length too large (default max is ${this.limits.maxDocumentBytes}). Got ${testSubmission.document.length}. Upsert may fail!`
      );
    if (
      JSON.stringify(testSubmission.metadata).length >
      this.limits.maxMetadataBytes
    )
      console.warn(
        `ChromaCloud::Metadata length too large (default max is ${this.limits.maxMetadataBytes}). Got ${JSON.stringify(testSubmission.metadata).length}. Upsert may fail!`
      );

    // If the submissions are not too large, just add them directly.
    if (submission.ids.length <= this.limits.maxRecordsPerWrite) {
      await collection.add(submission);
      return true;
    }

    this.logger(
      `Upsert Payload is too large (max is ${this.limits.maxRecordsPerWrite} records). Splitting into chunks of ${this.limits.maxRecordsPerWrite} records.`

View on GitHub (pinned to 3aec848f28)

Solutions

  1. Lower the workspace text-splitter chunk size (and overlap) so each chunk stays comfortably under 16,384 characters — e.g. chunk size 1000–4000 — then re-embed the document.
  2. If long chunks are intentional, use self-hosted Chroma (VECTOR_DB=chroma) or another vector DB without the per-document byte quota.
  3. Re-upload/re-embed the affected documents after changing chunk settings; existing oversized chunks are not auto-fixed.
  4. Watch the embed job logs: one oversized chunk warns but the rest of the batch continues — verify final document counts in the workspace.

Example fix

# before: workspace chunk size large enough to exceed the Chroma Cloud quota
workspace.textSplitterChunkSize = 20000  # chunk > 16,384 chars → "Document length too large"

# after
workspace.textSplitterChunkSize = 2000    # every chunk well under 16,384 chars
Defensive patterns

Strategy: validation

Validate before calling

const CHROMA_CLOUD_MAX_DOC_CHARS = 16_384;

const chunks = toChunks(text, workspace.chunkSize).map(chunk => {
  if (chunk.text && chunk.text.length > CHROMA_CLOUD_MAX_DOC_CHARS) {
    throw new Error(
      `Chunk of ${chunk.text.length} chars exceeds Chroma Cloud's 16,384-char document limit — lower the workspace chunk size.`
    );
  }
  return chunk;
});

Prevention

When it happens

Trigger: Embedding documents into a chromacloud workspace where a chunk's text exceeds 16,384 characters: a workspace text-splitter chunk size set very large, chunking effectively disabled for a pasted monolithic file, or HTML/scrape output whose cleaned text still yields huge chunks.

Common situations: Raising the workspace 'Text Chunk Size' preference far above defaults (e.g. 20000+) to 'keep more context', uploading a single giant text/PDF without splitting, migrating large-chunk workspaces from local Chroma (no byte limit) to Chroma Cloud, and non-ASCII content where characters≈bytes make the 16 KB budget tight.

Understand the failure class

Background: payload too large / request exceeds maximum size: why libraries cap bytes and how to fix oversize payloads — this error's family across 50 libraries.

Related errors


AI-assisted analysis of Mintplex-Labs/anything-llm@3aec848f28 (2026-08-18). Data as JSON: /api/errors/19666c50d817e52e. Report an issue: GitHub.