rohitg00/ai-engineering-from-scratch · error · ContractError

invalid JSON at character {exc.pos}

Error message

invalid JSON at character {exc.pos}

What it means

Raised by parse_and_validate when the raw string is not valid JSON; the character position from json.JSONDecodeError is wrapped into a ContractError at path '$'. Syntax errors are reported as structured validation issues rather than raw exceptions so callers get a uniform error shape.

Source

Thrown at certifications/claude/lessons/09-structured-output-and-defensive-parsing/code/main.py:44

TRIAGE_SCHEMA: dict[str, Any] = {
    "type": "object",
    "required": ["category", "priority", "summary", "needs_human"],
    "additionalProperties": False,
    "properties": {
        "category": {"type": "string", "enum": ["billing", "bug", "account", "other"]},
        "priority": {"type": "integer", "minimum": 1, "maximum": 5},
        "summary": {"type": "string", "minLength": 1, "maxLength": 240},
        "needs_human": {"type": "boolean"},
    },
}


def parse_and_validate(raw: str, schema: dict[str, Any]) -> Any:
    """Accept exactly one JSON value, then validate the supported schema subset."""
    try:
        value = json.loads(raw)
    except json.JSONDecodeError as exc:
        raise ContractError([ValidationIssue("$", f"invalid JSON at character {exc.pos}")]) from exc
    issues = validate(value, schema)
    if issues:
        raise ContractError(issues)
    return value


def validate(value: Any, schema: dict[str, Any], path: str = "$") -> list[ValidationIssue]:
    issues: list[ValidationIssue] = []
    expected = schema.get("type")
    if expected == "object":
        if not isinstance(value, dict):
            return [ValidationIssue(path, "expected object")]
        properties = schema.get("properties", {})
        for name in schema.get("required", []):
            if name not in value:
                issues.append(ValidationIssue(f"{path}.{name}", "required field is missing"))
        if schema.get("additionalProperties") is False:
            for name in value:

View on GitHub (pinned to 39ea8a1c6d)

Solutions

1. Prompt for raw JSON with no fences, or use forced tool/structured output so no prose is emitted
2. Extract the outermost JSON object from surrounding prose before parsing
3. Use the reported character position to locate and fix truncation or quote damage
4. If the cause is truncation, raise max_tokens and retry the generation

Example fix

# before
value = parse_and_validate("```json\n{\"a\": 1}\n```", schema)
# after
raw = '{"a": 1}'
value = parse_and_validate(raw, schema)

import json
def looks_like_json(raw: str) -> bool:
    s = raw.strip()
    if not s.startswith(("{", "[")):
        return False
    try:
        json.loads(s)
        return True
    except json.JSONDecodeError:
        return False

try:
    value = parse_and_validate(raw, schema)
except ContractError as exc:
    for issue in exc.issues:
        if issue.path == "$" and issue.message.startswith("invalid JSON"):
            raw = repair_or_reextract(raw)  # or re-prompt the model
            value = parse_and_validate(raw, schema)

When it happens

Trigger: Calling extract() or parse_and_validate() with malformed JSON: trailing commas, smart quotes, unescaped newlines in strings, markdown fences around the JSON, or a truncated model response.

Common situations: LLM output wrapped in ```json fences, model prose before or after the JSON, encoding damage from copy-paste, or a streamed response cut off mid-object.

Understand the failure class

• Parsing and encoding errors: unexpected token, malformed input — why parsers reject input and how to find the real culprit.

Related errors

• expected object
• tool_use requires name and object input
• response content must be a non-empty block list
• every content block needs a type
• event arrived after message_stop

AI-assisted analysis of rohitg00/ai-engineering-from-scratch@39ea8a1c6d (2026-08-26). Data as JSON: /api/errors/d51511678738b725. Report an issue: GitHub.