{"record":{"id":"81243b00a9531b5d","repo":"overleaf/overleaf","slug":"service-unavailable-shutdown-in-progress-res-send","errorCode":null,"errorMessage":"Service Unavailable (shutdown in progress res.sendStatus(503))","messagePattern":"Service Unavailable \\(shutdown in progress res\\.sendStatus\\(503\\)\\)","errorType":"http","errorClass":null,"httpStatus":503,"severity":"warning","filePath":"services/real-time/app.js","lineNumber":143,"sourceCode":"      return true\n    })\n  }\n})\n\n// Serve socket.io.js client file from imported dist folder\n// The express sendFile method correctly handles conditional\n// requests using the last-modified time and etag (which is\n// a combination of mtime and size)\nconst socketIOClientFolder = socketIOClient.dist\napp.get('/socket.io/socket.io.js', function (req, res) {\n  res.sendFile(Path.join(socketIOClientFolder, 'socket.io.min.js'))\n})\n\n// a 200 response on '/' is required for load balancer health checks\n// these operate separately from kubernetes readiness checks\napp.get('/', function (req, res) {\n  if (Settings.shutDownInProgress || DeploymentManager.deploymentIsClosed()) {\n    res.sendStatus(503) // Service unavailable\n  } else {\n    res.send('real-time is open')\n  }\n})\n\napp.get('/status', function (req, res) {\n  if (Settings.shutDownInProgress) {\n    res.sendStatus(503) // Service unavailable\n  } else {\n    res.send('real-time is alive')\n  }\n})\n\napp.get('/debug/events', function (req, res) {\n  Settings.debugEvents = parseInt(req.query.count, 10) || 20\n  logger.info({ count: Settings.debugEvents }, 'starting debug mode')\n  res.send(`debug mode will log next ${Settings.debugEvents} events`)\n})","sourceCodeStart":125,"sourceCodeEnd":161,"githubUrl":"https://github.com/overleaf/overleaf/blob/28ad3b03b71cb4311decdcb55c36b33ec10d72db/services/real-time/app.js#L125-L161","documentation":"The real-time service returns HTTP 503 on GET '/' when it is shutting down or the deployment is closed. Load balancer health checks hit this endpoint; a 503 tells them to stop routing new traffic to this instance during graceful shutdown or deployment drain.","triggerScenarios":"GET / while Settings.shutDownInProgress is true, or DeploymentManager.deploymentIsClosed() returns true (instance draining during a deploy/shutdown).","commonSituations":"Kubernetes or a load balancer polling '/' during a rolling restart; monitoring alerts firing because an instance reports 503 while being drained; misconfigured deployment manager closing deployments without traffic draining elsewhere.","solutions":["Confirm the instance is intentionally shutting down (rolling deploy/drain); 503 here is expected and transient","Check DeploymentManager.deploymentIsClosed() — if closed unintentionally, inspect deployment state/config","If 503 persists while shutDownInProgress is false, restart the service and verify shutdown handlers run","Exclude this instance from load balancer pools during deploys so users see no impact"],"exampleFix":"// before\nif (Settings.shutDownInProgress || DeploymentManager.deploymentIsClosed()) {\n  res.sendStatus(503)\n}\n// after\n// during shutdown, remove instance from LB pool and return 503 with a retry hint\nres.set('Retry-After', '5')\nres.sendStatus(503)","handlingStrategy":"retry","validationCode":"const isHealthy = async baseUrl => (await fetch(baseUrl + '/')).status === 200","typeGuard":null,"tryCatchPattern":"// retry with backoff while instance drains\nfor (let attempt = 0; attempt < 5; attempt++) {\n  const res = await fetch(url)\n  if (res.status !== 503) return res\n  await sleep(1000 * 2 ** attempt)\n}\nthrow new Error('real-time instance unavailable')","preventionTips":["Drain instances from the load balancer before shutdown so users never hit 503","Monitor deployment state if 503 persists after deploys finish","Set reasonable probe periods to avoid alert noise during rolling restarts"],"tags":["http-503","shutdown","load-balancer","health-check"],"backgroundTag":"service-unavailable-during-shutdown","analyzedSha":"28ad3b03b71cb4311decdcb55c36b33ec10d72db","analyzedAt":"2026-09-03T02:10:22.807Z","contentChangedAt":null,"schemaVersion":2},"datasetVersion":"2026-09-10T07:17:11.731Z"}