{"record":{"id":"a3a2e122a062e905","repo":"crewAIInc/crewAI","slug":"failed-to-evaluate-model-input-and-output-respons","errorCode":null,"errorMessage":"Failed to evaluate model input and output. Response status code: {response.status_code}. Reason: {response.text}","messagePattern":"Failed to evaluate model input and output\\. Response status code: (.+?)\\. Reason: (.+?)","errorType":"exception","errorClass":"Exception","httpStatus":null,"severity":"error","filePath":"lib/crewai-tools/src/crewai_tools/tools/patronus_eval_tool/patronus_eval_tool.py","lineNumber":152,"sourceCode":"            \"evaluated_model_retrieved_context\": evaluated_model_retrieved_context,\n            \"evaluators\": evals,\n        }\n\n        api_key = os.getenv(\"PATRONUS_API_KEY\", \"\")\n        headers = {\n            \"X-API-KEY\": api_key,\n            \"accept\": \"application/json\",\n            \"content-type\": \"application/json\",\n        }\n\n        response = requests.post(\n            self.evaluate_url,\n            headers=headers,\n            data=json.dumps(data),\n            timeout=30,\n        )\n        if response.status_code != 200:\n            raise Exception(\n                f\"Failed to evaluate model input and output. Response status code: {response.status_code}. Reason: {response.text}\"\n            )\n\n        return response.json()\n","sourceCodeStart":134,"sourceCodeEnd":157,"githubUrl":"https://github.com/crewAIInc/crewAI/blob/754d7323beb2fd042e33444a115ea2d5a47193f0/lib/crewai-tools/src/crewai_tools/tools/patronus_eval_tool/patronus_eval_tool.py#L134-L157","documentation":"Raised by PatronusEvaluationTool when the HTTP call to Patronus's /evaluate endpoint returns a non-200 status. The tool posts a JSON payload (evaluator config plus model input/output) with the API key header and a 30s timeout; any response other than 200 — bad key, malformed request, wrong evaluator name, server error — surfaces as this generic Exception with the status code and response body embedded.","triggerScenarios":"Calling the tool's evaluation with an invalid/expired Patronus API key (401/403), referencing a nonexistent evaluator id (4xx), sending a payload the API rejects (422), or Patronus returning a 5xx. Also triggered if the evaluate_url property points at a wrong endpoint.","commonSituations":"Expired API key, typo in the evaluator name, using a custom criteria payload with missing required fields, environment drift between staging and production API URLs, or transient 5xx/timeout issues from the Patronus service.","solutions":["Inspect response.text embedded in the message — it names the exact API-side reason (auth, validation, server error)","Verify the Patronus API key is valid and correctly passed (PATRONUS_API_KEY env var or constructor arg)","Check the evaluator specification (name/ID exists for your account, criteria fields are complete)","If the status is 5xx or rate-limit related, retry after a short backoff","Confirm evaluate_url matches the current Patronus API base URL documented for your account"],"exampleFix":"# before\ntool = PatronusEvaluationTool()\nresult = tool.run(...)  # raises Exception: status 401\n\n# after\nimport os\nassert os.environ.get(\"PATRONUS_API_KEY\"), \"set PATRONUS_API_KEY\"\nresult = tool.run(...)  # valid key -> 200\n","handlingStrategy":"retry","validationCode":"import os\n\ndef patronus_ready() -> bool:\n    return bool(os.environ.get(\"PATRONUS_API_KEY\"))","typeGuard":null,"tryCatchPattern":"import time\nfor attempt in range(3):\n    try:\n        result = tool.run(...)\n        break\n    except Exception as e:\n        msg = str(e)\n        if \"status code: 5\" in msg or \"status code: 429\" in msg:\n            time.sleep(2 ** attempt)\n            continue\n        raise  # 4xx auth/validation errors are not retryable","preventionTips":["Verify the API key at startup before running evaluations","Pin evaluator names to ones confirmed in the Patronus dashboard","Wrap calls in retry-with-backoff only for 5xx/429; escalate 401/422 immediately","Log response.text from the exception to speed up diagnosis"],"tags":["api","http","authentication","patronus","llm-evaluation"],"backgroundTag":null,"analyzedSha":"754d7323beb2fd042e33444a115ea2d5a47193f0","analyzedAt":"2026-08-15T04:06:56.746Z","schemaVersion":2},"datasetVersion":"2026-08-15T22:17:37.221Z"}