{"record":{"id":"83e71ad335a1c8ba","repo":"windmill-labs/windmill","slug":"http-agent-request-post-failed","errorCode":null,"errorMessage":"HTTP agent request POST {} failed {}","messagePattern":"HTTP agent request POST (.+?) failed (.+?)","errorType":"http","errorClass":null,"httpStatus":null,"severity":"error","filePath":"backend/windmill-common/src/worker.rs","lineNumber":583,"sourceCode":"    ) -> anyhow::Result<R> {\n        let base_url = self.base_internal_url.clone();\n\n        let response_builder = self.client.post(format!(\"{}{}\", base_url, url)).json(body);\n\n        let response_builder = match headers {\n            Some(headers) => response_builder.headers(headers),\n            None => response_builder,\n        };\n\n        let response = response_builder\n            .send()\n            .await\n            .map_err(|e| anyhow::anyhow!(e))?;\n        let status = response.status();\n        if status.is_success() {\n            Ok(response.json().await?)\n        } else {\n            Err(anyhow::anyhow!(format!(\n                \"HTTP agent request POST {} failed {}\",\n                url,\n                response.status()\n            )))\n        }\n    }\n\n    pub async fn get<R: DeserializeOwned>(&self, url: &str) -> anyhow::Result<R> {\n        let base_url = self.base_internal_url.clone();\n        let response = self\n            .client\n            .get(format!(\"{}{}\", base_url, url))\n            .send()\n            .await\n            .map_err(|e| anyhow::anyhow!(e))?;\n        let status = response.status();\n        if status.is_success() {\n            Ok(response.json().await?)","sourceCodeStart":565,"sourceCodeEnd":601,"githubUrl":"https://github.com/windmill-labs/windmill/blob/e474e8803ce2ff5c2df09a58dab51d45f5c922ca/backend/windmill-common/src/worker.rs#L565-L601","documentation":"Raised by HttpClient::post in windmill-common/src/worker.rs when a worker's HTTP-agent request (the Connection::Http path where a worker reaches the server API without direct DB access) returns a non-2xx status. The error message carries only the request path and the HTTP status code; the response body with the server's detailed error reason is discarded, so the status is all you get.","triggerScenarios":"Any worker-side POST through HttpClient::post (workspace service calls, global service calls, OpenAPI service calls, mock AI API, unauthed service calls, route helpers) where the server responds 4xx or 5xx: invalid token/permissions, nonexistent resource, malformed body, server error, or wrong base_internal_url hitting an unexpected endpoint.","commonSituations":"Misconfigured BASE_URL/base_internal_url pointing at the wrong server; worker token lacking workspace permissions; server unreachable behind a proxy returning 404/502; calling an endpoint gated behind an enterprise feature or disabled on the target instance; transient 5xx during backend restarts.","solutions":["Check the backend/API server logs for the matching request to get the real error body (the message only shows the status).","Verify the worker's base_internal_url / BASE_URL points at the correct reachable server.","Confirm the worker's token has the required workspace permissions for the endpoint.","Retry on 5xx statuses (transient server errors); investigate request payload if 4xx."],"exampleFix":"// before\nlet res: R = client.post(&url, headers, &body).await?;\n// after\nlet res: R = match client.post(&url, headers, &body).await {\n    Ok(r) => r,\n    Err(e) if e.to_string().contains(\"500\") || e.to_string().contains(\"502\") || e.to_string().contains(\"503\") => {\n        tokio::time::sleep(Duration::from_secs(2)).await;\n        client.post(&url, headers, &body).await?\n    }\n    Err(e) => return Err(e),\n};","handlingStrategy":"try-catch","validationCode":"// Rust: preflight reachability/health before the POST\nif let Err(e) = client.get::<serde_json::Value>(\"/health\").await {\n    return Err(anyhow::anyhow!(\"server unreachable before POST {url}: {e}\"));\n}","typeGuard":null,"tryCatchPattern":"match client.post::<_, R>(&url, headers, &body).await {\n    Ok(r) => r,\n    Err(e) => {\n        let msg = e.to_string();\n        if msg.contains(\"401\") || msg.contains(\"403\") {\n            return Err(anyhow::anyhow!(\"auth/permission failure on POST {url}: {msg}\"));\n        }\n        if msg.contains(\"500\") || msg.contains(\"502\") || msg.contains(\"503\") || msg.contains(\"504\") {\n            // retry transient server errors with backoff\n            tokio::time::sleep(Duration::from_secs(2)).await;\n            client.post::<_, R>(&url, headers, &body).await.map_err(|e| anyhow::anyhow!(\"POST {url} retry failed: {e}\"))?\n        } else {\n            return Err(anyhow::anyhow!(\"POST {url} rejected: {msg}\"));\n        }\n    }\n}","preventionTips":["Verify base_internal_url/BASE_URL during worker startup with a health check.","Grant the worker token only the permissions it needs, and test them before production jobs.","Add retry-with-backoff only for 5xx-class statuses, not 4xx.","Monitor server logs alongside worker logs so the discarded response body is visible.","Pin the server version/features the worker expects (feature-gated endpoints 404)."],"tags":["http","network","worker","rust"],"backgroundTag":"http-non-2xx-response","analyzedSha":"e474e8803ce2ff5c2df09a58dab51d45f5c922ca","analyzedAt":"2026-09-03T12:38:19.024Z","contentChangedAt":"2026-09-03T12:38:19.024Z","schemaVersion":2},"datasetVersion":"2026-09-08T10:18:20.063Z"}