{"record":{"id":"aaf184e4c1d2aa40","repo":"Lightning-AI/pytorch-lightning","slug":"the-server-didn-t-start-within-self-timeout-seco","errorCode":null,"errorMessage":"The server didn't start within {self.timeout} seconds.","messagePattern":"The server didn't start within (.+?) seconds\\.","errorType":"exception","errorClass":"Exception","httpStatus":null,"severity":"error","filePath":"src/lightning/pytorch/serve/servable_module_validator.py","lineNumber":111,"sourceCode":"\n        # Note: The Trainer needs to be detached from the pl_module before starting the process.\n        # This would fail during the deepcopy with DDP.\n        servable_module.trainer = None\n\n        process = Process(target=self._start_server, args=(servable_module, self.host, self.port, self.optimization))\n        process.start()\n\n        servable_module.trainer = trainer\n\n        ready = False\n        t0 = time.time()\n        while not ready:\n            with contextlib.suppress(requests.exceptions.ConnectionError):\n                resp = requests.get(f\"http://{self.host}:{self.port}/ping\")\n                ready = resp.status_code == 200\n            if time.time() - t0 > self.timeout:\n                process.kill()\n                raise Exception(f\"The server didn't start within {self.timeout} seconds.\")\n            time.sleep(0.1)\n\n        payload = servable_module.configure_payload()\n\n        if \"body\" not in payload:\n            raise Exception(f'Your provided payload {payload} should have a field named \"body\".')\n\n        self.resp = requests.post(f\"http://{self.host}:{self.port}/serve\", json=payload)\n        process.kill()\n\n        if is_overridden(\"configure_response\", servable_module, ServableModule):\n            response = servable_module.configure_response()\n            if self.resp.json() != response:\n                raise Exception(f\"The expected response {response} doesn't match the generated one {self.resp.json()}.\")\n\n        if self.exit_on_failure and not self.successful:\n            raise MisconfigurationException(\"The model isn't servable. Investigate the traceback and try again.\")\n","sourceCodeStart":93,"sourceCodeEnd":129,"githubUrl":"https://github.com/Lightning-AI/pytorch-lightning/blob/9fed5c27d2a62ff0efd6c3573599921d6ff67c14/src/lightning/pytorch/serve/servable_module_validator.py#L93-L129","documentation":"ServableModuleValidator spawns a subprocess running a FastAPI server on host:port and polls /ping until it returns 200. If the server is not ready within self.timeout seconds (default 30), the process is killed and a generic Exception is raised.","triggerScenarios":"Slow model load (large weights on slow disk), port already occupied so the server never binds, slow first import of torch/FastAPI in the subprocess, or an exception during server startup that prevents /ping from ever answering.","commonSituations":"CI machines with cold caches, another process holding the chosen port, firewalls blocking localhost connections, or an error inside the server subprocess whose traceback is only visible in subprocess logs.","solutions":["Increase the timeout: ServableModuleValidator(host=..., port=..., timeout=120)","Check the port is free (change port or kill the holder: lsof -i :PORT)","Watch subprocess logs for the real startup failure (the validator only reports the timeout)","Pre-warm heavy imports/weights so the subprocess starts faster"],"exampleFix":"// before\ntrainer = Trainer(callbacks=[ServableModuleValidator(host=\"127.0.0.1\", port=8000)])\n// after\ntrainer = Trainer(callbacks=[ServableModuleValidator(host=\"127.0.0.1\", port=8001, timeout=120)])","handlingStrategy":"retry","validationCode":"import socket\n\nwith socket.socket() as s:\n    s.bind((\"127.0.0.1\", 8000))  # raises if port taken — pick another port before training","typeGuard":null,"tryCatchPattern":"try:\n    trainer.fit(model)\nexcept Exception as e:\n    if \"didn't start within\" in str(e):\n        validator = ServableModuleValidator(host=h, port=p, timeout=120)\n        trainer = Trainer(callbacks=[validator], max_epochs=trainer.max_epochs)\n        trainer.fit(model)  # retry with longer timeout / different port\n    else:\n        raise","preventionTips":["Pass an explicit free port and generous timeout in CI","Keep model loads fast or pre-load weights outside the subprocess"],"tags":["lightning","serving","timeout","server-startup"],"backgroundTag":"server-startup-timeout","analyzedSha":"9fed5c27d2a62ff0efd6c3573599921d6ff67c14","analyzedAt":"2026-08-28T11:52:41.083Z","schemaVersion":2},"datasetVersion":"2026-08-28T16:17:29.566Z"}