crewAIInc/crewAI · error · Exception

An error occurred while testing the crew: {e}

Error message

An error occurred while testing the crew: {e}

What it means

Wrapper in the crew template's test() function. crew().test(n_iterations, eval_llm, inputs) runs the crew plus an evaluation pass with the given eval LLM; every failure — including int(sys.argv[1]) ValueError, missing sys.argv[2] IndexError, LLM errors, or eval-model misconfiguration — is re-raised as a generic Exception with 'An error occurred while testing the crew: {e}'.

Source

Thrown at lib/cli/src/crewai_cli/templates/crew/main.py:64

        {{crew_name}}().crew().replay(task_id=sys.argv[1])

    except Exception as e:
        raise Exception(f"An error occurred while replaying the crew: {e}")

def test():
    """
    Test the crew execution and returns the results.
    """
    inputs = {
        "topic": "AI LLMs",
        "current_year": str(datetime.now().year)
    }

    try:
        {{crew_name}}().crew().test(n_iterations=int(sys.argv[1]), eval_llm=sys.argv[2], inputs=inputs)

    except Exception as e:
        raise Exception(f"An error occurred while testing the crew: {e}")

def run_with_trigger():
    """
    Run the crew with trigger payload.
    """
    import json

    if len(sys.argv) < 2:
        raise Exception("No trigger payload provided. Please provide JSON payload as argument.")

    try:
        trigger_payload = json.loads(sys.argv[1])
    except json.JSONDecodeError:
        raise Exception("Invalid JSON payload provided as argument")

    inputs = {
        "crewai_trigger_payload": trigger_payload,
        "topic": "",

View on GitHub (pinned to 754d7323be)

Solutions

1. Supply both arguments: `uv run test 3 gpt-4o-mini` — missing the second arg is the most common cause.
2. Check the inner exception text after the colon for the underlying error.
3. Ensure the eval model name is valid and its API key is set in .env for the provider used.
4. Confirm the plain run (`uv run run`) works before testing.

Example fix

# before
try:
    MyCrew().crew().test(n_iterations=int(sys.argv[1]), eval_llm=sys.argv[2], inputs=inputs)
except Exception as e:
    raise Exception(f"An error occurred while testing the crew: {e}")

# after
if len(sys.argv) < 3:
    raise SystemExit("Usage: test <n_iterations> <eval_llm>")
MyCrew().crew().test(n_iterations=int(sys.argv[1]), eval_llm=sys.argv[2], inputs=inputs)

import sys
if len(sys.argv) < 3:
    raise SystemExit("Usage: test <n_iterations:int> <eval_llm>")
assert sys.argv[1].isdigit() and int(sys.argv[1]) > 0

def is_valid_test_args(argv: list[str]) -> bool:
    return len(argv) >= 3 and argv[1].isdigit() and int(argv[1]) > 0 and bool(argv[2])

try:
    MyCrew().crew().test(n_iterations=n, eval_llm=llm, inputs=inputs)
except Exception:
    logging.exception("test failed")
    raise

When it happens

Trigger: Running `uv run test <n> <eval_llm>` with a non-integer first arg, missing the eval_llm second arg, using an eval model string the installed provider cannot resolve (e.g. 'gpt-4o-mini' with no OpenAI key), or any kickoff failure during the test iterations.

Common situations: Developer runs `uv run test 3` and forgets the eval LLM; uses a model name that requires a key not present in .env; or the eval dependencies (crewai-tools / evaluation extras) are not installed in the UV environment.

Related errors

• An error occurred while running the crew: {e}
• An error occurred while training the crew: {e}
• An error occurred while replaying the crew: {e}
• No trigger payload provided. Please provide JSON payload as
• Invalid JSON payload provided as argument

AI-assisted analysis of crewAIInc/crewAI@754d7323be (2026-08-15). Data as JSON: /api/errors/773a22be6e59edd0. Report an issue: GitHub.