chroma-core/chroma · error · ChromaValueError

Expected IDs to be unique, but found duplicates of ${duplica

Error message

Expected IDs to be unique, but found duplicates of ${duplicates.join(", ")}

What it means

ChromaValueError thrown by validateIDs (utils.ts:219) when the ids array contains duplicates and there are 5 or fewer distinct duplicate values. Chroma ids are primary keys — re-adding the same id in one batch is ambiguous (insert-or-dedupe behavior is server policy), so the client lists the duplicated values in the message.

Source

Thrown at clients/new-js/packages/chromadb/src/utils.ts:219

    .filter(([id, _]) => typeof id !== "string")
    .map(([_, i]) => i);

  if (nonStrings.length > 0) {
    throw new ChromaValueError(
      `Found non-string IDs at ${nonStrings.join(", ")}`,
    );
  }

  const seen = new Set();
  const duplicates = ids.filter((id) => {
    if (seen.has(id)) {
      return id;
    }
    seen.add(id);
  });
  let message = "Expected IDs to be unique, but found duplicates of";
  if (duplicates.length > 0 && duplicates.length <= 5) {
    throw new ChromaValueError(`${message} ${duplicates.join(", ")}`);
  }
  if (duplicates.length > 0) {
    throw new ChromaValueError(
      `${message} ${duplicates.slice(0, 5).join(", ")}, ..., ${duplicates
        .slice(duplicates.length - 5)
        .join(", ")}`,
    );
  }
};

export const validateSparseVector = (v: unknown): v is SparseVector => {
  if (typeof v !== "object" || v === null) {
    return false;
  }

  const candidate = v as Record<string, unknown>;
  const indices = candidate.indices;
  const values = candidate.values;

View on GitHub (pinned to aecdd12c8a)

Solutions

Dedupe before sending: [...new Set(ids)] (aligning the row arrays accordingly)
Generate ids from a stable unique key (content hash, UUID) instead of a business field that can repeat
For true upserts, use collection.update() semantics or check existing ids via get() first

Example fix

// before
await collection.add({ ids, documents }); // ids has 'a' twice
// after
const seen = new Set();
const keep = ids.map(id => !seen.has(id) && seen.add(id));
await collection.add({ ids: ids.filter((_, i) => keep[i]), documents: documents.filter((_, i) => keep[i]) });

Defensive patterns

Strategy: validation

Validate before calling

function dedupeRows(ids, ...arrays) {
  const seen = new Set();
  const keep = ids.map(id => (seen.has(id) ? false : (seen.add(id), true)));
  return [ids.filter((_, i) => keep[i]), ...arrays.map(a => a.filter((_, i) => keep[i]))];
}

Type guard

const hasUniqueIds = (ids) => new Set(ids).size === ids.length;

Try / catch

try { await collection.add({ ids, documents }); } catch (e) { if (e instanceof ChromaValueError && /found duplicates of/.test(e.message)) { [ids, documents] = dedupeRows(ids, documents); await collection.add({ ids, documents }); } else throw e; }

Prevention

Dedupe ids (with aligned row arrays) before every add
Derive ids from a stable unique key or content hash
Split overlapping datasets before batching

When it happens

Trigger: collection.add({ ids: ['a','b','a'], documents }) — same id twice in one call; batches assembled from overlapping sources (retry + fresh rows); id generation keyed on a non-unique field like a category or date.

Common situations: Retried ingestion re-appending rows already built in this batch; ids derived from filenames that collide on case or truncation; union of two datasets without dedupe.

Related errors

AI-assisted analysis of chroma-core/chroma@aecdd12c8a (2026-08-16). Data as JSON: /api/errors/b9c86b514fe4c642. Report an issue: GitHub.