abi/screenshot-to-code · warning · HTTPException

Invalid {side} payload: {error}

Error message

Invalid {side} payload: {error}

What it means

400 raised by _load_openai_input_compare_payload in backend/routes/evals.py:192 when the JSON parses but compare_openai_inputs(payload, payload) raises ValueError — the decoded value is not a structurally valid OpenAI input payload (wrong shape/types for the comparison logic).

Source

Thrown at backend/routes/evals.py:192

    formatted: str


def _load_openai_input_compare_payload(raw_json: str, side: str) -> object:
    try:
        payload = json.loads(raw_json)
    except json.JSONDecodeError as error:
        raise HTTPException(
            status_code=400,
            detail=(
                f"Invalid {side} JSON: {error.msg} "
                f"(line {error.lineno}, column {error.colno})"
            ),
        )

    try:
        compare_openai_inputs(payload, payload)
    except ValueError as error:
        raise HTTPException(status_code=400, detail=f"Invalid {side} payload: {error}")

    return payload


@router.post("/openai-input-compare", response_model=OpenAIInputCompareResponse)
async def compare_openai_inputs_for_evals(
    request: OpenAIInputCompareRequest,
) -> OpenAIInputCompareResponse:
    left_payload = _load_openai_input_compare_payload(request.left_json, "left")
    right_payload = _load_openai_input_compare_payload(request.right_json, "right")
    comparison = compare_openai_inputs(left_payload, right_payload)

    difference = None
    if comparison.difference is not None:
        difference = OpenAIInputCompareDifferenceResponse(
            item_index=comparison.difference.item_index,
            path=comparison.difference.path,
            left_summary=comparison.difference.left_summary,

View on GitHub (pinned to d026163f58)

Solutions

1. Read the detail: the ValueError message names the structural problem.
2. Ensure both sides are actual OpenAI 'input' payloads (message arrays / content parts) extracted from request bodies, not whole envelopes.
3. Add a client-side shape check for the expected array/object structure before sending.

function isOpenAiInputPayload(v: unknown): v is Array<{ role: string; content: unknown }> {
  return Array.isArray(v) && v.every(
    (m) => typeof m === 'object' && m !== null && typeof (m as any).role === 'string'
  );
}

if (res.status === 400 && detail.startsWith('Invalid left payload') || detail.startsWith('Invalid right payload')) {
  // structural mismatch: show which side failed and the reason
}

When it happens

Trigger: POST /openai-input-compare where left_json/right_json is valid JSON but not an OpenAI input object — e.g. a bare string, number, array of wrong element types, or an object missing required fields.

Common situations: Comparing generic JSON that happens to be lying around; schema drift after an OpenAI API change; comparing the full request envelope instead of the inner input value.

Related errors

• Invalid {side} JSON: {error.msg} (line {error.lineno}, colum
• Invalid run id
• max_age_days must be >= 1
• Design system name is required
• Invalid eval set name: {set_name!r}

AI-assisted analysis of abi/screenshot-to-code@d026163f58 (2026-08-14). Data as JSON: /api/errors/d7cfb13000ae44ec. Report an issue: GitHub.