{"record":{"id":"ed40d11cde23e410","repo":"t8y2/dbx","slug":"agent-request-capacity-is-temporarily-exhausted","errorCode":null,"errorMessage":"agent request capacity is temporarily exhausted","messagePattern":"agent request capacity is temporarily exhausted","errorType":"exception","errorClass":null,"httpStatus":null,"severity":"warning","filePath":"agents/drivers/tdengine/src/runtime.rs","lineNumber":112,"sourceCode":"                Ok(request) => handle_request(runtime.clone(), request).await,\n                Err(error) => error_response(Value::Null, \"request\", None, error.into()),\n            };\n            responses.send(response).map_err(|_| anyhow!(\"TDengine response writer stopped\"))?;\n            while requests.join_next().await.is_some() {}\n            break;\n        }\n        let request_permit = if parsed.as_ref().is_ok_and(|request| is_capacity_exempt(&request.method)) {\n            None\n        } else {\n            match runtime.request_slots.clone().try_acquire_owned() {\n                Ok(permit) => Some(permit),\n                Err(_) => {\n                    let response = match parsed {\n                        Ok(request) => error_response(\n                            if request.id.is_null() { json!(1) } else { request.id },\n                            &request.method,\n                            session_id(&request.params),\n                            anyhow!(\"agent request capacity is temporarily exhausted\"),\n                        ),\n                        Err(error) => error_response(Value::Null, \"request\", None, error.into()),\n                    };\n                    responses.send(response).map_err(|_| anyhow!(\"TDengine response writer stopped\"))?;\n                    continue;\n                }\n            }\n        };\n        let runtime = runtime.clone();\n        let responses = responses.clone();\n        requests.spawn(async move {\n            let _request_permit = request_permit;\n            let response = match parsed {\n                Ok(request) => handle_request(runtime, request).await,\n                Err(error) => error_response(Value::Null, \"request\", None, error.into()),\n            };\n            let _ = responses.send(response);\n        });","sourceCodeStart":94,"sourceCodeEnd":130,"githubUrl":"https://github.com/t8y2/dbx/blob/c0390bff16418b651f4728520d99adf8ce48829a/agents/drivers/tdengine/src/runtime.rs#L94-L130","documentation":"The runtime bounds non-control work with a Semaphore of MAX_CONCURRENT_REQUESTS permits (runtime.rs:153). When every permit is taken — i.e. MAX_CONCURRENT_REQUESTS requests are already in flight — any new non-exempt request is immediately rejected with this error instead of being queued. Control methods (handshake, cancel_session, close_session, disconnect) are exempt via is_capacity_exempt so clients can always free capacity. The error is surfaced as a structured RpcResponse with category 'resource', retryable=true, session_disposition 'keep', so the session survives.","triggerScenarios":"Sending more concurrent requests than MAX_CONCURRENT_REQUESTS without waiting for responses — e.g. issuing many execute_query/list_tables calls in parallel, or long-running queries (get_explain_info, execute_transaction) holding permits while more requests stream in on stdin.","commonSituations":"A client pipeline fanning out metadata discovery across many tables at once; parallel dashboard queries against a busy agent; a stuck/slow query that never finishes keeps holding permits so subsequent requests get rejected; batching scripts with no concurrency cap.","solutions":["Reduce client-side concurrency below MAX_CONCURRENT_REQUESTS (serialize or limit parallel in-flight requests)","Retry the request after in-flight ones complete — the error is explicitly marked retryable=true with session kept","Check for hung long-running queries and reclaim permits via cancel_session (which is capacity-exempt)","Await each response before issuing the next request if you rely on strict sequential protocol"],"exampleFix":"// before: unbounded fan-out\nfor table in tables {\n  send({\"method\":\"list_objects\",\"params\":{...}}); // may exceed request capacity\n}\n// after: bounded pipeline (e.g. 4 at a time)\nfor chunk in tables.chunks(4) {\n  let responses = join_all(chunk.map(|t| send(list_objects(t)))).await;\n  for r in responses { handle(r); } // releases permits before next chunk\n}","handlingStrategy":"retry","validationCode":"let inFlight = 0;\nconst MAX_CONCURRENT = 8; // keep at or below the agent's MAX_CONCURRENT_REQUESTS\nfunction canSend() { return inFlight < MAX_CONCURRENT; }\nif (!canSend()) await waitForSlot();","typeGuard":"function isCapacityError(res) {\n  return res?.error?.data?.category === \"resource\"\n    && /request capacity/i.test(res.error?.message ?? \"\");\n}","tryCatchPattern":"async function sendWithRetry(request, retries = 5) {\n  for (let i = 0; i < retries; i++) {\n    const res = await sendRpc(request);\n    if (isCapacityError(res)) {\n      await sleep(backoff(i)); // exponential backoff; session is kept\n      continue;\n    }\n    return res;\n  }\n  throw new Error(\"agent still at request capacity after retries\");\n}","preventionTips":["Bound client-side concurrency to the agent's MAX_CONCURRENT_REQUESTS before fanning out","Expedite stuck work via cancel_session (capacity-exempt) instead of letting hung queries hold permits","Prefer the structured error's retryable=true flag: back off and retry, the session is kept","Await each response before issuing the next request when strict ordering matters"],"tags":["backpressure","concurrency","capacity","tdengine","retryable"],"backgroundTag":"request-capacity-exhausted","analyzedSha":"c0390bff16418b651f4728520d99adf8ce48829a","analyzedAt":"2026-09-05T23:05:10.900Z","contentChangedAt":"2026-09-05T23:05:10.900Z","schemaVersion":2},"datasetVersion":"2026-09-14T05:17:10.506Z"}