{"record":{"id":"ab73fd875536c0af","repo":"hcengineering/platform","slug":"high-hung-sessions","errorCode":null,"errorMessage":"high hung sessions","messagePattern":"high hung sessions","errorType":"console","errorClass":null,"httpStatus":null,"severity":"error","filePath":"foundations/server/packages/server/src/sessionManager.ts","lineNumber":543,"sourceCode":"        const sec = Math.round((now - r.start) / 1000)\n        if (sec > hangRequestTimeoutSeconds) {\n          hungRequests += 1\n        }\n        totalRequests += 1\n      }\n\n      const hungRequestsPercent = totalRequests > 0 ? (100 * hungRequests) / totalRequests : 0\n      if (hungRequestsPercent > this.hungRequestsFailPercent) {\n        hungSessions += 1\n      }\n    }\n\n    this.ctx.measure('sessions-hung', hungSessions, true)\n\n    const hungSessionsPercent = totalSessions > 0 ? (100 * hungSessions) / totalSessions : 0\n\n    if (hungSessionsPercent > this.hungSessionsFailPercent) {\n      this.ctx.warn('high hung sessions', { hungSessionsPercent })\n      return 'unhealthy'\n    }\n\n    if (hungSessionsPercent > this.hungSessionsWarnPercent) {\n      this.ctx.warn('high degraded sessions', { hungSessionsPercent })\n      return 'degraded'\n    }\n\n    return 'healthy'\n  }\n\n  tickCounter = 0\n\n  @withContext('🧭 get-workspace')\n  async getWorkspace (\n    ctx: MeasureContext,\n    workspaceUuid: WorkspaceUuid,\n    workspaceInfo: LoginInfoWorkspace | undefined,","sourceCodeStart":525,"sourceCodeEnd":561,"githubUrl":"https://github.com/hcengineering/platform/blob/63e28dc96483967b2fc21c881b3f1023c1de7718/foundations/server/packages/server/src/sessionManager.ts#L525-L561","documentation":"Health check computes the percentage of hung sessions versus total sessions; if it exceeds hungSessionsFailPercent the server logs this warning and reports itself 'unhealthy' (a warn-level threshold reports 'degraded'). This is the instance-level health signal consumed by load balancers/orchestrators to stop routing traffic to a sick node.","triggerScenarios":"checkHealth (invoked by health, getStatistics, status endpoints) measures hungSessions and finds hungSessionsPercent > hungSessionsFailPercent — many clients stopped responding or many requests are stuck.","commonSituations":"Network outage on the client side leaving many dead sockets; server-side dependency outage causing mass request hangs; load balancer not draining connections; too-strict hungSessionsFailPercent for workloads with legitimately slow clients.","solutions":["Investigate cause of mass hangs: check 'session hang, closing...' and 'request hang found' logs and downstream service health","Restart/replace the unhealthy instance so the orchestrator reschedules traffic","Verify downstream dependencies (DB, storage, Kafka) are reachable from the server","Tune hungSessionsFailPercent/hungSessionsWarnPercent thresholds to match realistic client behavior"],"exampleFix":"// before\nhealth probe: strict 90% fail threshold, no LB drain\n// after\nWORKSPACE_HUNG_SESSIONS_FAIL_PERCENT=50\n# and configure LB to respect /health status before rerouting","handlingStrategy":"validation","validationCode":"const stats = await fetch(`${serverUrl}/status`).then(r => r.json())\nif (stats.hungSessionsPercent > 50) {\n  console.error('node unhealthy, rerouting traffic')\n}","typeGuard":"function isUnhealthy(h: { hungSessionsPercent: number; failPercent: number }): boolean {\n  return h.hungSessionsPercent > h.failPercent\n}","tryCatchPattern":"try {\n  const health = await getHealth()\n  if (health === 'unhealthy') {\n    await drainAndRestart()\n  }\n} catch (err) {\n  logger.error('health check failed', { err })\n}","preventionTips":["Wire /health into load balancer and orchestrator probes","Keep client pings flowing so sessions don't accumulate as hung","Monitor downstream dependencies whose outages cause mass hangs","Tune hungSessionsFailPercent/WarnPercent to realistic workload baselines"],"tags":["health","sessions","monitoring"],"backgroundTag":"high-hung-sessions","analyzedSha":"63e28dc96483967b2fc21c881b3f1023c1de7718","analyzedAt":"2026-08-29T15:21:27.377Z","schemaVersion":2},"datasetVersion":"2026-08-29T17:17:51.833Z"}