{"record":{"id":"6d61dc6fb5947da4","repo":"instructure/canvas-lms","slug":"dropped-pandapub-notification-for-channel","errorCode":null,"errorMessage":"dropped pandapub notification for #{channel}","messagePattern":"dropped pandapub notification for #(.+?)","errorType":"console","errorClass":null,"httpStatus":null,"severity":"warning","filePath":"gems/canvas_panda_pub/lib/canvas_panda_pub/client.rb","lineNumber":86,"sourceCode":"    #\n    #  channel - A String representing the PandaPub channel to post to. It should\n    #    not include the application id, as that will be added by the library.\n    #  payload - A Hash with the payload. It will be converted to JSON with JSON.dump.\n\n    def post_update(channel, payload)\n      path = \"/channel/#{@application_id}#{channel}\"\n      request = Net::HTTP::Post.new(path, {\n                                      \"Content-Type\" => \"application/json\"\n                                    })\n      request.basic_auth @key_id, @key_secret\n\n      body = JSON.dump(payload)\n\n      http = Net::HTTP.new(@uri.host, @uri.port)\n      http.use_ssl = (@uri.scheme == \"https\")\n\n      unless @worker.push(channel, proc { http.request(request, body) })\n        @logger.warn(\"dropped pandapub notification for #{channel}\")\n      end\n    end\n\n    # Generate a token for subscribing to a channel.\n    #\n    # channel - A String with the channel to be subscribed to. Don't include the application\n    #   id.\n    # read - true if this token should allow reading from the channel.\n    # write - true if this token should allow posting to the channel.\n    # expires - A Date object specifying when the token should expire.\n    #\n    # Returns a String token.\n\n    def generate_token(channel, read: false, write: false, expires: 1.hour.from_now)\n      JSON::JWT.new({\n                      keyId: @key_id,\n                      channel: \"/#{@application_id}#{channel}\",\n                      pub: write,","sourceCodeStart":68,"sourceCodeEnd":104,"githubUrl":"https://github.com/instructure/canvas-lms/blob/1c9f0bb8013ed69c4f2efe11fd483025469b7e6c/gems/canvas_panda_pub/lib/canvas_panda_pub/client.rb#L68-L104","documentation":"CanvasPandaPub::Client#post_update publishes a notification by pushing a proc onto a background worker queue. When the worker's queue is full or shut down, push returns false (or nil) and the client logs this warning and silently drops the notification — the HTTP request is never sent to PandaPub.","triggerScenarios":"@worker.push(channel, proc { http.request(request, body) }) returns falsy — the worker queue is saturated (max queue size reached) or the worker thread is not running (shutdown/not started) — while post_update attempts to publish to channel.","commonSituations":"Publishing bursts faster than the single worker thread can drain the queue; PandaPub endpoint slow/unreachable causing queue backlog; app shutting down while live-event updates are still being published; worker never initialized in some boot paths.","solutions":["Increase the worker queue size or drain rate if drops occur under normal load","Check PandaPub endpoint health/latency — slow pushes starve the queue","Add retry/fallback (e.g., synchronous publish or re-enqueue) for critical notifications instead of accepting the drop","Monitor the warning rate to detect saturation before messages are lost in production"],"exampleFix":"// before\nunless @worker.push(channel, proc { http.request(request, body) })\n  @logger.warn(\"dropped pandapub notification for #{channel}\")\nend\n// after\nunless @worker.push(channel, proc { http.request(request, body) })\n  @logger.warn(\"dropped pandapub notification for #{channel}; publishing synchronously\")\n  http.request(request, body)\nend","handlingStrategy":"fallback","validationCode":"# check worker health before publishing\nunless CanvasPandaPub.worker&.alive?\n  Rails.logger.warn('pandapub worker down; using fallback channel')\nend","typeGuard":null,"tryCatchPattern":"unless worker.push(channel, proc { http.request(request, body) })\n  logger.warn(\"dropped pandapub notification for #{channel}\")\n  fallback_publish(channel, payload) # e.g., synchronous send or delayed job\nend","preventionTips":["Monitor drop warnings as a saturation signal","Keep the PandaPub endpoint fast and healthy so the queue drains","Use a durable fallback (delayed job) for critical live-event notifications"],"tags":["websocket","pubsub","queue","panda-pub"],"backgroundTag":"network-request-failed","analyzedSha":"1c9f0bb8013ed69c4f2efe11fd483025469b7e6c","analyzedAt":"2026-09-15T20:33:18.891Z","contentChangedAt":"2026-09-15T20:33:18.891Z","schemaVersion":2},"datasetVersion":"2026-09-23T02:17:17.105Z"}