{"record":{"id":"bb31e5c2fed50d3e","repo":"abi/screenshot-to-code","slug":"unexpected-response-from-model-result","errorCode":null,"errorMessage":"Unexpected response from {model}: {result}","messagePattern":"Unexpected response from (.+?): (.+?)","errorType":"exception","errorClass":"ValueError","httpStatus":null,"severity":"error","filePath":"backend/run_image_generation_evals.py","lineNumber":137,"sourceCode":"def extract_output_url(result: Any, model: ReplicateEvalModel) -> str:\n    if isinstance(result, str):\n        return result\n\n    if isinstance(result, dict):\n        url = cast(dict[str, Any], result).get(\"url\")\n        if isinstance(url, str) and url:\n            return url\n\n    if isinstance(result, list) and result:\n        first = cast(list[Any], result)[0]\n        if isinstance(first, str) and first:\n            return first\n        if isinstance(first, dict):\n            url = cast(dict[str, Any], first).get(\"url\")\n            if isinstance(url, str) and url:\n                return url\n\n    raise ValueError(f\"Unexpected response from {model}: {result}\")\n\n\nasync def generate_image(\n    model: ReplicateEvalModel, prompt: str, api_key: str\n) -> str:\n    result = await call_replicate_model(\n        MODEL_PATHS[model],\n        build_replicate_input(model, prompt),\n        api_key,\n    )\n    return extract_output_url(result, model)\n\n\nasync def download_image(\n    session: aiohttp.ClientSession, url: str, path: Path\n) -> int:\n    async with session.get(url) as response:\n        response.raise_for_status()","sourceCodeStart":119,"sourceCodeEnd":155,"githubUrl":"https://github.com/abi/screenshot-to-code/blob/d026163f586dfa8c5c10d28c36edd59a9d3b0e88/backend/run_image_generation_evals.py#L119-L155","documentation":"ValueError(\"Unexpected response from {model}: {result}\") raised by extract_output_url() when the Replicate prediction output does not match any shape it knows: a string URL, a non-empty list of strings, or a list of dicts containing a \"url\" string. Different Replicate image models return different output schemas (single URL, URL list, file-object lists with different keys), so adding a model or a version bump that changes the schema breaks extraction. The message embeds the raw result, which is the key diagnostic.","triggerScenarios":"Running run_image_generation_evals.py with flux_2_klein or z_image_turbo (or a new entry in MODEL_PATHS) whose prediction output is, e.g., a dict like {\"images\": [...]}, a list of file objects keyed by something other than \"url\", or a data URI.","commonSituations":"Replicate model version updated upstream changing the output schema; a new model added to MODEL_PATHS/build_replicate_input without extending extract_output_url; output being None because the prediction was still running when polled.","solutions":["Copy the raw result from the error message and identify where the URL lives (which key / nesting)","Extend extract_output_url with a branch for that shape (see exampleFix)","Pin/verify the exact model version in MODEL_PATHS so schema does not drift","If result is None/empty, ensure the prediction is fully completed before extraction (poll until status == \"succeeded\")"],"exampleFix":"# before\n    raise ValueError(f\"Unexpected response from {model}: {result}\")\n\n# after (handle dict-with-images and file objects)\n    if isinstance(result, dict):\n        images = result.get(\"images\")\n        if isinstance(images, list) and images:\n            first = images[0]\n            url = first.get(\"url\") if isinstance(first, dict) else first\n            if isinstance(url, str) and url:\n                return url\n    raise ValueError(f\"Unexpected response from {model}: {result}\")","handlingStrategy":"fallback","validationCode":"def known_output_shape(result: Any) -> bool:\n    if isinstance(result, str) and result:\n        return True\n    if isinstance(result, list) and result:\n        first = result[0]\n        if isinstance(first, str) and first:\n            return True\n        if isinstance(first, dict) and isinstance(first.get(\"url\"), str):\n            return True\n    return False  # extract_output_url would raise ValueError","typeGuard":null,"tryCatchPattern":"try:\n    url = extract_output_url(result, model)\nexcept ValueError:\n    url = extract_from_raw(result)  # fallback: dig URL out of the printed shape, log it, file an issue to update the extractor","preventionTips":["Pin exact Replicate model versions in MODEL_PATHS so output schema cannot drift silently","Log raw prediction outputs in eval runs so new shapes are diagnosable","When adding a model, add its output shape branch to extract_output_url in the same change"],"tags":["replicate","api-contract","evals","schema-drift"],"backgroundTag":null,"analyzedSha":"d026163f586dfa8c5c10d28c36edd59a9d3b0e88","analyzedAt":"2026-08-14T22:02:06.951Z","schemaVersion":2},"datasetVersion":"2026-08-15T22:17:37.221Z"}