{"record":{"id":"773a22be6e59edd0","repo":"crewAIInc/crewAI","slug":"an-error-occurred-while-testing-the-crew-e","errorCode":null,"errorMessage":"An error occurred while testing the crew: {e}","messagePattern":"An error occurred while testing the crew: (.+?)","errorType":"exception","errorClass":"Exception","httpStatus":null,"severity":"error","filePath":"lib/cli/src/crewai_cli/templates/crew/main.py","lineNumber":64,"sourceCode":"        {{crew_name}}().crew().replay(task_id=sys.argv[1])\n\n    except Exception as e:\n        raise Exception(f\"An error occurred while replaying the crew: {e}\")\n\ndef test():\n    \"\"\"\n    Test the crew execution and returns the results.\n    \"\"\"\n    inputs = {\n        \"topic\": \"AI LLMs\",\n        \"current_year\": str(datetime.now().year)\n    }\n\n    try:\n        {{crew_name}}().crew().test(n_iterations=int(sys.argv[1]), eval_llm=sys.argv[2], inputs=inputs)\n\n    except Exception as e:\n        raise Exception(f\"An error occurred while testing the crew: {e}\")\n\ndef run_with_trigger():\n    \"\"\"\n    Run the crew with trigger payload.\n    \"\"\"\n    import json\n\n    if len(sys.argv) < 2:\n        raise Exception(\"No trigger payload provided. Please provide JSON payload as argument.\")\n\n    try:\n        trigger_payload = json.loads(sys.argv[1])\n    except json.JSONDecodeError:\n        raise Exception(\"Invalid JSON payload provided as argument\")\n\n    inputs = {\n        \"crewai_trigger_payload\": trigger_payload,\n        \"topic\": \"\",","sourceCodeStart":46,"sourceCodeEnd":82,"githubUrl":"https://github.com/crewAIInc/crewAI/blob/754d7323beb2fd042e33444a115ea2d5a47193f0/lib/cli/src/crewai_cli/templates/crew/main.py#L46-L82","documentation":"Wrapper in the crew template's test() function. crew().test(n_iterations, eval_llm, inputs) runs the crew plus an evaluation pass with the given eval LLM; every failure — including int(sys.argv[1]) ValueError, missing sys.argv[2] IndexError, LLM errors, or eval-model misconfiguration — is re-raised as a generic Exception with 'An error occurred while testing the crew: {e}'.","triggerScenarios":"Running `uv run test <n> <eval_llm>` with a non-integer first arg, missing the eval_llm second arg, using an eval model string the installed provider cannot resolve (e.g. 'gpt-4o-mini' with no OpenAI key), or any kickoff failure during the test iterations.","commonSituations":"Developer runs `uv run test 3` and forgets the eval LLM; uses a model name that requires a key not present in .env; or the eval dependencies (crewai-tools / evaluation extras) are not installed in the UV environment.","solutions":["Supply both arguments: `uv run test 3 gpt-4o-mini` — missing the second arg is the most common cause.","Check the inner exception text after the colon for the underlying error.","Ensure the eval model name is valid and its API key is set in .env for the provider used.","Confirm the plain run (`uv run run`) works before testing."],"exampleFix":"# before\ntry:\n    MyCrew().crew().test(n_iterations=int(sys.argv[1]), eval_llm=sys.argv[2], inputs=inputs)\nexcept Exception as e:\n    raise Exception(f\"An error occurred while testing the crew: {e}\")\n\n# after\nif len(sys.argv) < 3:\n    raise SystemExit(\"Usage: test <n_iterations> <eval_llm>\")\nMyCrew().crew().test(n_iterations=int(sys.argv[1]), eval_llm=sys.argv[2], inputs=inputs)","handlingStrategy":"validation","validationCode":"import sys\nif len(sys.argv) < 3:\n    raise SystemExit(\"Usage: test <n_iterations:int> <eval_llm>\")\nassert sys.argv[1].isdigit() and int(sys.argv[1]) > 0","typeGuard":"def is_valid_test_args(argv: list[str]) -> bool:\n    return len(argv) >= 3 and argv[1].isdigit() and int(argv[1]) > 0 and bool(argv[2])","tryCatchPattern":"try:\n    MyCrew().crew().test(n_iterations=n, eval_llm=llm, inputs=inputs)\nexcept Exception:\n    logging.exception(\"test failed\")\n    raise","preventionTips":["Provide both iteration count and eval model name","Confirm the eval model string is resolvable with your configured provider key","Get a plain `run` passing before `test`"],"tags":["crewai","cli","template","evaluation"],"backgroundTag":null,"analyzedSha":"754d7323beb2fd042e33444a115ea2d5a47193f0","analyzedAt":"2026-08-15T04:06:56.746Z","schemaVersion":2},"datasetVersion":"2026-08-15T22:17:37.221Z"}