datawhalechina/hello-agents · error · ValueError

缺少必需字段: problem 或 answer

Error message

缺少必需字段: problem 或 answer

What it means

ValueError raised by the AIME-style problem generator after parsing the LLM's JSON output: the parsed object lacks the required 'problem' or 'answer' key. It fires only after json.loads succeeded, so the model returned syntactically valid JSON with the wrong schema.

Source

Thrown at code/chapter12/data_generation/aime_generator.py:216

            # 方法：先将字符串中的单个反斜杠替换为双反斜杠（但保留已经转义的）
            # 这样LaTeX的 \frac 会变成 \\frac，在JSON中是合法的

            # 使用正则表达式：找到所有未转义的反斜杠（不是\\的\）
            # 并将其替换为\\
            fixed_json_str = re.sub(r'(?<!\\)\\(?!["\\/bfnrtu])', r'\\\\', json_str)

            try:
                problem_data = json.loads(fixed_json_str)
            except json.JSONDecodeError:
                # 如果还是失败，打印错误信息并抛出
                print(f"❌ JSON解析失败:")
                print(f"原始响应: {response[:500]}...")
                print(f"提取的JSON: {json_str[:500]}...")
                raise

        # 验证必需字段
        if "problem" not in problem_data or "answer" not in problem_data:
            raise ValueError("缺少必需字段: problem 或 answer")

        # 验证答案范围
        answer = int(problem_data.get("answer", 0))
        if not (0 <= answer <= 999):
            print(f"⚠️ 答案超出范围: {answer}，调整为0-999范围内")
            answer = max(0, min(999, answer))
            problem_data["answer"] = answer

        # 确保有默认值
        problem_data.setdefault("solution", "No solution provided")
        problem_data.setdefault("topic", "Uncategorized")

        return problem_data

    def _get_default_problem(self) -> Dict[str, Any]:
        """获取默认题目（生成失败时使用）"""
        return {
            "problem": "生成失败，请重新生成",

View on GitHub (pinned to 606a07d341)

Solutions

  1. Inspect the printed '原始响应'/'提取的JSON' output to see exactly which keys the model produced
  2. Strengthen the prompt: show an explicit JSON schema and an example with keys problem/answer, and state that other keys are optional but these two are mandatory
  3. Add a repair pass: on missing keys, re-ask the model with the invalid JSON and the error before giving up
  4. Pin a stronger model or lower temperature for this generation step

Example fix

# before
if "problem" not in problem_data or "answer" not in problem_data:
    raise ValueError("缺少必需字段: problem 或 answer")

# after: one retry with schema feedback, then fail
if "problem" not in problem_data or "answer" not in problem_data:
    problem_data = retry_with_schema_feedback(response)
    if "problem" not in problem_data or "answer" not in problem_data:
        raise ValueError("缺少必需字段: problem 或 answer")
Defensive patterns

Strategy: validation

Validate before calling

REQUIRED = ('problem', 'answer')

def is_valid_problem(d) -> bool:
    return isinstance(d, dict) and all(k in d for k in REQUIRED)

Type guard

def is_valid_problem(d) -> bool:
    return (
        isinstance(d, dict)
        and isinstance(d.get('problem'), str)
        and str(d.get('answer', '')).lstrip('-').isdigit()
    )

Try / catch

try:
    problem_data = parse_and_validate(response)
except ValueError:
    problem_data = regenerate_with_schema_feedback(response)  # one repair pass
    if not is_valid_problem(problem_data):
        raise

Prevention

When it happens

Trigger: The LLM returns {"question": ..., "ans": ...} or nested/wrapped objects instead of the flat {problem, answer, ...} schema; the model wraps data in markdown or prose that survived the earlier extraction but shifted keys; weak model or truncated prompt not showing the required schema.

Common situations: Switching to a smaller/cheaper model that ignores the JSON schema; prompt template drift after refactoring; temperature too high producing creative key names; few-shot examples out of sync with the validation code.

Related errors

AI-assisted analysis of datawhalechina/hello-agents@606a07d341 (2026-08-14). Data as JSON: /api/errors/37ea0944c85c3d98. Report an issue: GitHub.