{"record":{"id":"95c675996f33de44","repo":"weaviate/weaviate","slug":"too-many-shards-in-request-d-d","errorCode":null,"errorMessage":"too many shards in request (%d > %d)","messagePattern":"too many shards in request \\((.+?) > (.+?)\\)","errorType":"http","errorClass":null,"httpStatus":400,"severity":"error","filePath":"adapters/handlers/rest/clusterapi/indices_replicas.go","lineNumber":1219,"sourceCode":"\treturn 0, nil\n}\n\nfunc (i *replicatedIndices) postAsyncCheckpoint() http.Handler {\n\treturn http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {\n\t\targs := regxAsyncCheckpoint.FindStringSubmatch(r.URL.Path)\n\t\tif len(args) != 2 {\n\t\t\thttp.Error(w, \"invalid URI\", http.StatusBadRequest)\n\t\t\treturn\n\t\t}\n\t\tclassName := args[1]\n\n\t\tvar req asyncCheckpointCreateRequest\n\t\tif status, err := readAsyncCheckpointBody(w, r, &req); err != nil {\n\t\t\thttp.Error(w, err.Error(), status)\n\t\t\treturn\n\t\t}\n\t\tif len(req.Shards) > replica.AsyncCheckpointMaxShardsPerRequest {\n\t\t\thttp.Error(w, fmt.Sprintf(\"too many shards in request (%d > %d)\",\n\t\t\t\tlen(req.Shards), replica.AsyncCheckpointMaxShardsPerRequest),\n\t\t\t\thttp.StatusBadRequest)\n\t\t\treturn\n\t\t}\n\t\tif req.CutoffMs <= 0 {\n\t\t\thttp.Error(w, \"cutoff_ms must be > 0\", http.StatusBadRequest)\n\t\t\treturn\n\t\t}\n\t\tif req.CreatedAtMs <= 0 {\n\t\t\thttp.Error(w, \"created_at_ms must be > 0\", http.StatusBadRequest)\n\t\t\treturn\n\t\t}\n\n\t\tcreatedAt := time.UnixMilli(req.CreatedAtMs).UTC()\n\t\t// Past values are fine (tie-breaker handles them); reject only far-future skew.\n\t\tif skew := time.Until(createdAt); skew > replica.AsyncCheckpointCreatedAtSkewTolerance {\n\t\t\thttp.Error(w,\n\t\t\t\tfmt.Sprintf(\"created_at_ms is too far in the future (%s ahead of this node's clock; tolerance %s)\",","sourceCodeStart":1201,"sourceCodeEnd":1237,"githubUrl":"https://github.com/weaviate/weaviate/blob/75aa4b6d11f8818305aafd4440b4e32794f7ca04/adapters/handlers/rest/clusterapi/indices_replicas.go#L1201-L1237","documentation":"The async checkpoint create handler caps how many shard names a single request may carry (replica.AsyncCheckpointMaxShardsPerRequest). Exceeding it returns 400 with \"too many shards in request (%d > %d)\" from adapters/handlers/rest/clusterapi/indices_replicas.go:1219, bounding per-request fan-out.","triggerScenarios":"POST to /replicas/indices/<class>/async-checkpoint whose JSON body's shards array contains more entries than replica.AsyncCheckpointMaxShardsPerRequest (also indirectly enforced by the 64 KiB body cap).","commonSituations":"Collections with very many shards where the broadcaster sends all shards at once; scripts checkpointing every shard in a single call; oversized batches after shard-count growth (e.g. dynamic sharding scaled up).","solutions":["Split the shard list into batches of at most replica.AsyncCheckpointMaxShardsPerRequest and issue multiple POSTs.","Update the coordinating client/broadcaster to chunk shard lists automatically.","If the limit is genuinely too low for your shard count, evaluate raising AsyncCheckpointMaxShardsPerRequest in a patched build."],"exampleFix":"// before\npostCheckpoint(class, allShards) // len(allShards) > max\n// after\nfor batch := range slices.Chunk(allShards, replica.AsyncCheckpointMaxShardsPerRequest) { postCheckpoint(class, batch) }","handlingStrategy":"validation","validationCode":"const maxShards = replica.AsyncCheckpointMaxShardsPerRequest\nif len(shards) > maxShards {\n\tshards = shards[:maxShards] // or split into batches\n}","typeGuard":"func withinShardLimit(shards []string) bool { return len(shards) <= replica.AsyncCheckpointMaxShardsPerRequest }","tryCatchPattern":"if resp.StatusCode == 400 && strings.Contains(body, \"too many shards in request\") {\n\treturn postCheckpointInBatches(shards) // resend in compliant batches\n}","preventionTips":["Chunk shard lists client-side against AsyncCheckpointMaxShardsPerRequest before posting.","Alert on shard-count growth so batch sizes are revisited when collections scale.","Document the request cap wherever checkpoint tooling is built."],"tags":["validation","weaviate","async-replication","batching"],"backgroundTag":"request-limit-exceeded","analyzedSha":"75aa4b6d11f8818305aafd4440b4e32794f7ca04","analyzedAt":"2026-09-04T14:58:20.392Z","contentChangedAt":"2026-09-04T14:58:20.392Z","schemaVersion":2},"datasetVersion":"2026-09-11T21:17:09.523Z"}