What causes "Backoff and retry" in chroma-core/chroma?

Catch ChromaBackoffError and retry with exponential backoff plus jitter; honor any wait guidance in the message.

chroma-core/chroma · warning · ChromaBackoffError

Backoff and retry

Error message

Backoff and retry

What it means

Thrown by chromaFetch (chroma-fetch.ts:134) as a ChromaBackoffError when the server returns 429 with error === 'Backoff'. Chroma Cloud uses this signal for server-side throttling with an advisory component: the caller should stop hammering the endpoint and retry later, respecting any guidance in the message (e.g. suggested wait).

Source

Thrown at clients/new-js/packages/chromadb/src/chroma-fetch.ts:134

        ) {
          throw new ChromaQuotaExceededError(body?.message);
        }
        throw new ChromaClientError(body?.message || "Unprocessable Entity");
      } catch (error) {
        if (
          error instanceof ChromaQuotaExceededError ||
          error instanceof ChromaClientError
        ) {
          throw error;
        }
        throw new ChromaClientError(
          `Unprocessable Entity: ${response.statusText}`,
        );
      }
    case 429:
      const rateLimitBody = await getErrorBody(response);
      if (rateLimitBody.error === "Backoff") {
        throw new ChromaBackoffError(
          rateLimitBody.message || "Backoff and retry",
        );
      }
      throw new ChromaRateLimitError("Rate limit exceeded");
  }

  const errorMessage = await getErrorMessage(response);
  throw new ChromaServerError(errorMessage);
};

View on GitHub (pinned to aecdd12c8a)

Solutions

Catch ChromaBackoffError and retry with exponential backoff plus jitter; honor any wait guidance in the message.
Throttle the producer (p-limit, token bucket) so issuance stays under the throttle threshold.
Reduce concurrency of workers sharing one API key/tenant.
If sustained throughput is required, contact Chroma Cloud about raising the tenant's limits.

Example fix

// before
const results = await Promise.all(docs.map(d => collection.add([d]))); // 429 Backoff

// after
import pLimit from "p-limit";
const limit = pLimit(5);
await Promise.all(docs.map(d => limit(() => collection.add([d]))));
// plus: on ChromaBackoffError, sleep with exponential backoff + jitter before retrying

Defensive patterns

Strategy: retry

Try / catch

for (let attempt = 0; ; attempt++) {
  try {
    return await operation();
  } catch (e) {
    if (e instanceof ChromaBackoffError && attempt < 8) {
      await new Promise(r => setTimeout(r, Math.min(2 ** attempt * 500, 30_000) + Math.random() * 500));
      continue;
    }
    throw e;
  }
}

Prevention

Run all Cloud calls through a single retry helper with exponential backoff + jitter.
Limit concurrency (p-limit) and pace producers below throttle thresholds.
Instrument ChromaBackoffError rates to detect undersized throughput limits early.

When it happens

Trigger: Sustained high-rate ingestion or querying against Chroma Cloud where the service sheds load; fan-out workers all retrying simultaneously after a slowdown; bursts exceeding the tenant's throughput class.

Common situations: Bulk backfills without rate limiting; retry storms after a transient slowdown; too many concurrent workers on one tenant.

Related errors

AI-assisted analysis of chroma-core/chroma@aecdd12c8a (2026-08-16). Data as JSON: /api/errors/ba137f09f0023722. Report an issue: GitHub.