Internal Server Error (redis/pubsub health check failed res.

Error message

Internal Server Error (redis/pubsub health check failed res.sendStatus(500))

What it means

GET /health_check returns HTTP 500 when either the redis client healthCheck fails or HealthCheckManager reports accumulated pub/sub errors. The real-time service depends on redis pub/sub to fan out messages, so a broken connection makes the instance ineligible for traffic.

Source

Thrown at services/real-time/app.js:169

    res.sendStatus(503) // Service unavailable
  } else {
    res.send('real-time is alive')
  }
})

app.get('/debug/events', function (req, res) {
  Settings.debugEvents = parseInt(req.query.count, 10) || 20
  logger.info({ count: Settings.debugEvents }, 'starting debug mode')
  res.send(`debug mode will log next ${Settings.debugEvents} events`)
})

const rclient = redis.createClient(Settings.redis.realtime)

function healthCheck(req, res) {
  rclient.healthCheck(function (error) {
    if (error) {
      logger.err({ err: error }, 'failed redis health check')
      res.sendStatus(500)
    } else if (HealthCheckManager.isFailing()) {
      const status = HealthCheckManager.status()
      logger.err({ pubSubErrors: status }, 'failed pubsub health check')
      res.sendStatus(500)
    } else {
      res.sendStatus(200)
    }
  })
}
app.get(
  '/health_check',
  (req, res, next) => {
    if (Settings.shutDownComplete) {
      return res.sendStatus(503)
    }
    next()
  },
  healthCheck

View on GitHub (pinned to 28ad3b03b7)

Solutions

Check redis connectivity and credentials in Settings.redis.realtime; verify with redis-cli ping from the pod
Inspect logs for 'failed redis health check' or 'failed pubsub health check' for the underlying error
If HealthCheckManager.isFailing(), look at recent pub/sub errors and reconnect logic
Scale/replace the pod and confirm redis is sized correctly for the cluster

Example fix

// before
rclient.healthCheck(function (error) {
  if (error) { res.sendStatus(500) }
})
// after
// surface the cause in the response body for faster triage
rclient.healthCheck(function (error) {
  if (error) {
    res.status(500).json({ error: 'redis-unreachable', detail: error.message })
  }
})

Defensive patterns

Strategy: retry

Validate before calling

const redisUp = await new Promise((ok, bad) => rclient.ping(e => (e ? bad(e) : ok(true))))

Try / catch

try {
  const res = await fetch(healthCheckUrl)
  if (res.status === 500) {
    logger.error('real-time redis/pubsub unhealthy; retrying')
    return retryWithBackoff()
  }
} catch (err) {
  logger.warn({ err }, 'health check request failed')
}

Prevention

Monitor redis latency and connection counts from real-time pods
Alert on HealthCheckManager pub/sub error accumulation before it flips the endpoint
Pin and test redis credentials/connection settings per environment
Set redis client reconnect strategy so transient outages self-heal

When it happens

Trigger: rclient.healthCheck callback receives an error (redis unreachable/auth failure), or HealthCheckManager.isFailing() is true because pub/sub message processing is erroring.

Common situations: Redis restart or failover in the cluster; wrong Settings.redis.realtime host/port/password; network policy blocking the pod from redis; persistent pub/sub handler exceptions flipping HealthCheckManager into failing state.

Understand the failure class

HTTP status errors: handling 4xx and 5xx responses — how to handle 4xx and 5xx responses properly.

Related errors

AI-assisted analysis of overleaf/overleaf@28ad3b03b7 (2026-09-03). Data as JSON: /api/errors/62f11320557e552b. Report an issue: GitHub.