abi/screenshot-to-code · error · ValueError

Unexpected response from {model}: {result}

Error message

Unexpected response from {model}: {result}

What it means

ValueError("Unexpected response from {model}: {result}") raised by extract_output_url() when the Replicate prediction output does not match any shape it knows: a string URL, a non-empty list of strings, or a list of dicts containing a "url" string. Different Replicate image models return different output schemas (single URL, URL list, file-object lists with different keys), so adding a model or a version bump that changes the schema breaks extraction. The message embeds the raw result, which is the key diagnostic.

Source

Thrown at backend/run_image_generation_evals.py:137

def extract_output_url(result: Any, model: ReplicateEvalModel) -> str:
    if isinstance(result, str):
        return result

    if isinstance(result, dict):
        url = cast(dict[str, Any], result).get("url")
        if isinstance(url, str) and url:
            return url

    if isinstance(result, list) and result:
        first = cast(list[Any], result)[0]
        if isinstance(first, str) and first:
            return first
        if isinstance(first, dict):
            url = cast(dict[str, Any], first).get("url")
            if isinstance(url, str) and url:
                return url

    raise ValueError(f"Unexpected response from {model}: {result}")


async def generate_image(
    model: ReplicateEvalModel, prompt: str, api_key: str
) -> str:
    result = await call_replicate_model(
        MODEL_PATHS[model],
        build_replicate_input(model, prompt),
        api_key,
    )
    return extract_output_url(result, model)


async def download_image(
    session: aiohttp.ClientSession, url: str, path: Path
) -> int:
    async with session.get(url) as response:
        response.raise_for_status()

View on GitHub (pinned to d026163f58)

Solutions

  1. Copy the raw result from the error message and identify where the URL lives (which key / nesting)
  2. Extend extract_output_url with a branch for that shape (see exampleFix)
  3. Pin/verify the exact model version in MODEL_PATHS so schema does not drift
  4. If result is None/empty, ensure the prediction is fully completed before extraction (poll until status == "succeeded")

Example fix

# before
    raise ValueError(f"Unexpected response from {model}: {result}")

# after (handle dict-with-images and file objects)
    if isinstance(result, dict):
        images = result.get("images")
        if isinstance(images, list) and images:
            first = images[0]
            url = first.get("url") if isinstance(first, dict) else first
            if isinstance(url, str) and url:
                return url
    raise ValueError(f"Unexpected response from {model}: {result}")
Defensive patterns

Strategy: fallback

Validate before calling

def known_output_shape(result: Any) -> bool:
    if isinstance(result, str) and result:
        return True
    if isinstance(result, list) and result:
        first = result[0]
        if isinstance(first, str) and first:
            return True
        if isinstance(first, dict) and isinstance(first.get("url"), str):
            return True
    return False  # extract_output_url would raise ValueError

Try / catch

try:
    url = extract_output_url(result, model)
except ValueError:
    url = extract_from_raw(result)  # fallback: dig URL out of the printed shape, log it, file an issue to update the extractor

Prevention

When it happens

Trigger: Running run_image_generation_evals.py with flux_2_klein or z_image_turbo (or a new entry in MODEL_PATHS) whose prediction output is, e.g., a dict like {"images": [...]}, a list of file objects keyed by something other than "url", or a data URI.

Common situations: Replicate model version updated upstream changing the output schema; a new model added to MODEL_PATHS/build_replicate_input without extending extract_output_url; output being None because the prediction was still running when polled.

Related errors

AI-assisted analysis of abi/screenshot-to-code@d026163f58 (2026-08-14). Data as JSON: /api/errors/bb31e5c2fed50d3e. Report an issue: GitHub.