{"record":{"id":"aade68d8a3b32a6e","repo":"mlflow/mlflow","slug":"test-cases-at-indices-missing-goal-indices-must","errorCode":null,"errorMessage":"Test cases at indices {missing_goal_indices} must have 'goal' field","messagePattern":"Test cases at indices (.+?) must have 'goal' field","errorType":"validation","errorClass":"ValueError","httpStatus":null,"severity":"error","filePath":"mlflow/genai/simulators/simulator.py","lineNumber":533,"sourceCode":"                    \"EvaluationDataset passed to ConversationSimulator must contain \"\n                    \"conversational test cases with a 'goal' field in the 'inputs' column\"\n                )\n            return records\n\n        if isinstance(test_cases, DataFrame):\n            return test_cases.to_dict(\"records\")\n\n        return test_cases\n\n    def _validate_test_cases(self, test_cases: list[dict[str, Any]]) -> None:\n        if not test_cases:\n            raise ValueError(\"test_cases cannot be empty\")\n\n        missing_goal_indices = [\n            i for i, test_case in enumerate(test_cases) if not test_case.get(\"goal\")\n        ]\n        if missing_goal_indices:\n            raise ValueError(f\"Test cases at indices {missing_goal_indices} must have 'goal' field\")\n\n        indices_with_invalid_context = [\n            i\n            for i, test_case in enumerate(test_cases)\n            if not (\n                isinstance(test_case.get(\"context\"), dict)\n                or _is_missing_context_value(test_case.get(\"context\"))\n            )\n        ]\n        if indices_with_invalid_context:\n            raise ValueError(\n                f\"Test cases at indices {indices_with_invalid_context} must have 'context' as \"\n                \"a dict when provided.\"\n            )\n\n        indices_with_reserved_context_keys = [\n            i\n            for i, test_case in enumerate(test_cases)","sourceCodeStart":515,"sourceCodeEnd":551,"githubUrl":"https://github.com/mlflow/mlflow/blob/6a27f2decc0b76eb1b54af31849784addb357dbc/mlflow/genai/simulators/simulator.py#L515-L551","documentation":"Every test case must include a truthy 'goal' field describing what the simulated user wants to accomplish. _validate_test_cases collects the indices of cases where test_case.get('goal') is falsy (missing, None, or empty string) and raises this ValueError listing them.","triggerScenarios":"Setting ConversationSimulator.test_cases to a list where one or more dicts lack 'goal' or have goal=None/'' — e.g. [{'goal': 'x'}, {'persona': 'admin'}] fails at index 1.","commonSituations":"Merging test case sets from different sources where some lack 'goal'; typos like 'goals' or 'objective'; records built by LLM distillation where goal extraction failed and returned None.","solutions":["Add a non-empty 'goal' string to every test case dict at the reported indices","Fix key typos so the field is exactly 'goal'","Filter or repair records with falsy goals before constructing the simulator"],"exampleFix":"// before\ncases = [{'goal': 'Ask about refunds'}, {'persona': 'angry customer'}]\n// after\ncases = [{'goal': 'Ask about refunds', 'persona': 'angry customer'}]","handlingStrategy":"validation","validationCode":"bad = [i for i, c in enumerate(test_cases) if not (isinstance(c, dict) and c.get('goal'))]\nif bad:\n    raise ValueError(f'Cases {bad} need a non-empty goal')","typeGuard":"def has_goal(case: dict) -> bool:\n    return isinstance(case.get('goal'), str) and case['goal'].strip() != ''","tryCatchPattern":"try:\n    sim = ConversationSimulator(test_cases=cases)\nexcept ValueError as e:\n    if \"must have 'goal' field\" in str(e):\n        import re\n        bad = re.findall(r'\\[([^\\]]+)\\]', str(e))\n        for i in ast.literal_eval(bad[0]):\n            cases[i]['goal'] = default_goal\n    else:\n        raise","preventionTips":["Validate every case has a non-empty 'goal' before assignment","Avoid typos: use a constant GOAL = 'goal' as the key","LLM-distilled goals should be checked for None/empty before use"],"tags":["python","validation","missing-field"],"backgroundTag":"missing-required-field","analyzedSha":"6a27f2decc0b76eb1b54af31849784addb357dbc","analyzedAt":"2026-08-29T20:54:51.419Z","schemaVersion":2},"datasetVersion":"2026-08-29T22:17:34.462Z"}