{"record":{"id":"af408334eddf0545","repo":"mlflow/mlflow","slug":"test-cases-at-indices-indices-with-invalid-contex","errorCode":null,"errorMessage":"Test cases at indices {indices_with_invalid_context} must have 'context' as a dict when provided.","messagePattern":"Test cases at indices (.+?) must have 'context' as a dict when provided\\.","errorType":"validation","errorClass":"ValueError","httpStatus":null,"severity":"error","filePath":"mlflow/genai/simulators/simulator.py","lineNumber":544,"sourceCode":"        if not test_cases:\n            raise ValueError(\"test_cases cannot be empty\")\n\n        missing_goal_indices = [\n            i for i, test_case in enumerate(test_cases) if not test_case.get(\"goal\")\n        ]\n        if missing_goal_indices:\n            raise ValueError(f\"Test cases at indices {missing_goal_indices} must have 'goal' field\")\n\n        indices_with_invalid_context = [\n            i\n            for i, test_case in enumerate(test_cases)\n            if not (\n                isinstance(test_case.get(\"context\"), dict)\n                or _is_missing_context_value(test_case.get(\"context\"))\n            )\n        ]\n        if indices_with_invalid_context:\n            raise ValueError(\n                f\"Test cases at indices {indices_with_invalid_context} must have 'context' as \"\n                \"a dict when provided.\"\n            )\n\n        indices_with_reserved_context_keys = [\n            i\n            for i, test_case in enumerate(test_cases)\n            if isinstance(test_case.get(\"context\"), dict)\n            and set(test_case[\"context\"]) & _RESERVED_CONTEXT_KEYS\n        ]\n        if indices_with_reserved_context_keys:\n            raise ValueError(\n                f\"Test cases at indices {indices_with_reserved_context_keys} have context keys \"\n                f\"that conflict with keys reserved by ConversationSimulator \"\n                f\"({_RESERVED_CONTEXT_KEYS}). These keys are used to inject conversation \"\n                \"history ('input', 'messages') or session ID ('mlflow_session_id'). \"\n                \"Rename the conflicting keys in the test case context.\"\n            )","sourceCodeStart":526,"sourceCodeEnd":562,"githubUrl":"https://github.com/mlflow/mlflow/blob/6a27f2decc0b76eb1b54af31849784addb357dbc/mlflow/genai/simulators/simulator.py#L526-L562","documentation":"The optional 'context' field of a test case, when provided and not a missing-value sentinel (None, NaN, etc.), must be a dict mapping variable names to values used for prompt templating. _validate_test_cases raises this ValueError listing indices whose context is a non-dict value (e.g. a string or list).","triggerScenarios":"Setting ConversationSimulator.test_cases where some case has context='product X' or context=['a','b'] instead of a dict — via ConversationSimulator(test_cases=...) or sim.test_cases = ....","commonSituations":"Copying context from YAML/JSON configs where it is a flat string; assuming context is free-form like a chat history list; pandas records conversion turning dicts into NaN mixes.","solutions":["Wrap the context value in a dict, e.g. context={'product': 'MLflow tracing'}","Remove the context key entirely if not needed (None/missing is allowed)","Normalize string contexts to {'input': value} or another named variable"],"exampleFix":"// before\n{'goal': 'Ask about limits', 'context': 'free tier user'}\n// after\n{'goal': 'Ask about limits', 'context': {'plan': 'free tier'}}","handlingStrategy":"type-guard","validationCode":"bad = [i for i, c in enumerate(test_cases)\n       if 'context' in c and c['context'] is not None and not isinstance(c['context'], dict)]\nif bad:\n    raise ValueError(f'Cases {bad}: context must be a dict')","typeGuard":"def has_valid_context(case: dict) -> bool:\n    ctx = case.get('context')\n    return ctx is None or isinstance(ctx, dict)","tryCatchPattern":"try:\n    sim = ConversationSimulator(test_cases=cases)\nexcept ValueError as e:\n    if \"must have 'context' as a dict\" in str(e):\n        for c in cases:\n            if c.get('context') is not None and not isinstance(c['context'], dict):\n                c['context'] = {'value': c['context']}\n    else:\n        raise","preventionTips":["Always model context as dict[str, Any] in your data generation","Parse YAML/JSON context strings into dicts before building test cases","Add a schema check (pydantic/dataclass) for test case records"],"tags":["python","validation","type-mismatch"],"backgroundTag":"schema-validation-failed","analyzedSha":"6a27f2decc0b76eb1b54af31849784addb357dbc","analyzedAt":"2026-08-29T20:54:51.419Z","schemaVersion":2},"datasetVersion":"2026-08-29T22:17:34.462Z"}