Lightning-AI/pytorch-lightning · error · Exception

The expected response {response} doesn't match the generated

Error message

The expected response {response} doesn't match the generated one {self.resp.json()}.

What it means

If the model overrides configure_response, the validator compares the actual HTTP response JSON with the model's declared expected response. A mismatch (different values, keys, or JSON-serialized types) raises a generic Exception listing both dicts.

Source

Thrown at src/lightning/pytorch/serve/servable_module_validator.py:125

                resp = requests.get(f"http://{self.host}:{self.port}/ping")
                ready = resp.status_code == 200
            if time.time() - t0 > self.timeout:
                process.kill()
                raise Exception(f"The server didn't start within {self.timeout} seconds.")
            time.sleep(0.1)

        payload = servable_module.configure_payload()

        if "body" not in payload:
            raise Exception(f'Your provided payload {payload} should have a field named "body".')

        self.resp = requests.post(f"http://{self.host}:{self.port}/serve", json=payload)
        process.kill()

        if is_overridden("configure_response", servable_module, ServableModule):
            response = servable_module.configure_response()
            if self.resp.json() != response:
                raise Exception(f"The expected response {response} doesn't match the generated one {self.resp.json()}.")

        if self.exit_on_failure and not self.successful:
            raise MisconfigurationException("The model isn't servable. Investigate the traceback and try again.")

        if self.successful:
            _logger.info(f"Your model is servable and the received payload was {self.resp.json()}.")

    @property
    def successful(self) -> Optional[bool]:
        """Returns whether the model was successfully served."""
        return self.resp.status_code == 200 if self.resp else None

    @override
    def state_dict(self) -> dict[str, Any]:
        return {"successful": self.successful, "optimization": self.optimization, "server": self.server}

    @staticmethod
    def _start_server(servable_model: ServableModule, host: str, port: int, _: bool) -> None:

View on GitHub (pinned to 9fed5c27d2)

Solutions

1. Run once without overriding configure_response, inspect self.resp.json() in logs, then set configure_response to exactly that structure
2. Compare key sets and serialized types (lists vs numbers) — not tensor objects
3. Use approximations (round/pytest.approx-style tolerance) instead of exact equality if floats differ

Example fix

// before
def configure_response(self):
    return {"output": tensor([1.0, 2.0])}  # tensor, not JSON-serializable form
// after
def configure_response(self):
    return {"output": [1.0, 2.0]}

from lightning.pytorch.utilities import is_overridden
from lightning.pytorch.serve import ServableModule

if is_overridden("configure_response", model, ServableModule):
    expected = model.configure_response()
    assert isinstance(expected, dict), "expected response must be a JSON-compatible dict"

try:
    trainer.fit(model)
except Exception as e:
    if "doesn't match the generated one" in str(e):
        # log self.resp.json() structure and align configure_response
        print(e)

When it happens

Trigger: Overriding configure_response with hardcoded expected values that don't match serve_step output after serialization (e.g. tensors serialized to nested lists, float precision differences, extra/missing keys).

Common situations: Updating model logic but forgetting to update configure_response; expecting a tensor repr while the serializer returned a Python list; dtype/rounding differences between local run and served run.

Related errors

• The `configure_payload` method needs to be overridden.
• The `configure_serialization` method needs to be overridden.
• The `serve_step` method needs to be overridden.
• The server didn't start within {self.timeout} seconds.
• Your provided payload {payload} should have a field named "b

AI-assisted analysis of Lightning-AI/pytorch-lightning@9fed5c27d2 (2026-08-28). Data as JSON: /api/errors/a174bc67424e9e31. Report an issue: GitHub.