Tencent/WeKnora · error

opensearch: batch size exceeds driver cap

Error message

opensearch: batch size exceeds driver cap

What it means

Sentinel ErrBatchTooLarge: Save/Delete rejected before contacting OpenSearch because the bulk request exceeded a driver-side sync limit — either estimated body over 10MB (given vector dims) or more than 1000 documents in one batch. It guards cluster memory and load, not a server failure.

Source

Thrown at internal/application/repository/retriever/opensearch/errors.go:49

	// ErrVersionUnsupported — cluster is not OpenSearch, is OS 1.x, or is
	// OS 2.0~2.3 (pre-Lucene-HNSW-GA). probeVersion enforces.
	ErrVersionUnsupported = errors.New("opensearch: cluster version unsupported")

	// ErrConfigInvalid — IndexConfig / storeID / sanitizeIndexName guard
	// failed, or the k-NN plugin is missing on one or more cluster nodes.
	ErrConfigInvalid = errors.New("opensearch: invalid index config")

	// ErrFeatureNotEnabled — stubs.go returns this from methods whose real
	// implementation has not landed yet (CopyIndices / BatchUpdateChunk* /
	// swapToVersion, plus the read/write methods that a follow-up commit
	// will replace with production code).
	ErrFeatureNotEnabled = errors.New("opensearch: feature not enabled in this build")

	// ErrBatchTooLarge — Save / Delete batch exceeded the driver's sync
	// cap. Distinct from ErrFeatureNotEnabled so the service layer can
	// chunk + retry rather than treat the failure as "waiting on a future
	// implementation."
	ErrBatchTooLarge = errors.New("opensearch: batch size exceeds driver cap")

	// ErrCircuitBreaker — OpenSearch k-NN circuit breaker returned 429
	// (knn_circuit_breaker_exception). Classified as transient so the
	// caller can retry after the operator scales the cluster.
	ErrCircuitBreaker = errors.New("opensearch: knn circuit breaker open")
)

// isTransientErr classifies sentinel errors. Transient errors do not get
// persisted in ensureReady's initErr cache — they can be retried by the
// next caller after the underlying cause clears.
func isTransientErr(err error) bool {
	return errors.Is(err, ErrTransport) || errors.Is(err, ErrCircuitBreaker)
}

View on GitHub (pinned to 988cbb0330)

Solutions

1. Split the batch into chunks of at most 1000 documents (and below the 10MB estimated body cap) and retry each chunk
2. Reduce vector dimensions or payload size per document, or stream items in smaller groups instead of one bulk call

When it happens

Trigger: Thrown at internal/application/repository/retriever/opensearch/errors.go:49 when the library encounters an invalid state.

Common situations: See trigger scenarios.

AI-assisted analysis of Tencent/WeKnora@988cbb0330 (2026-09-02). Data as JSON: /api/errors/90a2cd88be4e5fcd. Report an issue: GitHub.