instructure/canvas-lms · critical · MaxNumberOfClientsReachedError

max number of clients reached

Error message

max number of clients reached

What it means

canvas_cache wraps redis-client operations so that redis's 'max number of clients reached' CommandError is translated into a MaxNumberOfClientsReachedError with the original message. Redis itself emits the error when the server's maxclients limit is hit and new connections are refused. The wrapper only re-raises when the message contains the exact phrase.

Solutions

1. Increase Redis server maxclients (redis.conf) and/or the OS file-descriptor limit (ulimit -n)
2. Fix connection leaks: ensure Redis clients/pools are reused and connections are released
3. Restart or CLIENT KILL stale connections to drop the current client count below maxclients
4. Scale horizontally with Redis Cluster or split workloads across instances; add retry/backoff on MaxNumberOfClientsReachedError

Example fix

// before
redis.set(key, value) # raw, no retry
// after
begin
  redis.set(key, value)
rescue CanvasCache::Redis::MaxNumberOfClientsReachedError
  sleep_with_backoff && retry
end

begin
  info = redis.client.call('INFO', 'clients')
  connected = info[/connected_clients:(\d+)/, 1].to_i
  at_capacity = connected >= max_clients_threshold
rescue StandardError
  at_capacity = false
end

begin
  redis.call('GET', key)
rescue CanvasCache::Redis::MaxNumberOfClientsReachedError
  sleep(backoff *= 2)
  retry
end

When it happens

Trigger: Any Redis command or connection attempt (connect, call, call_pipelined) while the Redis server has hit its maxclients limit; the underlying CommandError message includes 'max number of clients reached'.

Common situations: Connection leaks in an app (clients never returned to pool) pushing the server to maxclients; multiple app servers/shards sharing a small maxclients setting; stuck client connections from long-running jobs.

Understand the failure class

Background: ECONNREFUSED and "connection refused" / "could not connect to server" errors: what they mean and how to fix them — this error's family across 44 libraries.

Related errors

• all slots filled
• an object with an interface for loading settings must be…
• CANVAS_HTTP CB_TRIP ON #
• Redis method `# ` is not supported by Twemproxy, and so…
• Redis method `# ` is potentially dangerous, and should only…

AI-assisted analysis of instructure/canvas-lms@1c9f0bb801 (2026-09-15). Data as JSON: /api/errors/3c06683166afe385. Report an issue: GitHub.

Appendix: source

Thrown at gems/canvas_cache/lib/redis_client/max_clients.rb:40

  MaxNumberOfClientsReachedError = Class.new(ConnectionError)

  module MaxClients
    def connect(*)
      protect_from_max_clients { super }
    end

    def call(*)
      protect_from_max_clients { super }
    end

    def call_pipelined(*)
      protect_from_max_clients { super }
    end

    def protect_from_max_clients
      yield
    rescue CommandError => e
      raise MaxNumberOfClientsReachedError, e.message if e.message.include?("max number of clients reached")

      raise
    end
  end
end

View on GitHub (pinned to 1c9f0bb801)