{"record":{"id":"6b52c05043e129e7","repo":"amir20/dozzle","slug":"cloud-dispatcher-rate-limited-retry-after-s","errorCode":null,"errorMessage":"cloud dispatcher rate limited, retry after %s","messagePattern":"cloud dispatcher rate limited, retry after (.+?)","errorType":"exception","errorClass":null,"httpStatus":null,"severity":"warning","filePath":"internal/notification/dispatcher/cloud.go","lineNumber":75,"sourceCode":"// API key). Retrying won't help until the user fixes their key, which recreates the\n// dispatcher and resets the breaker.\nconst unauthorizedRetryAfter = 6 * time.Hour\n\n// ResetBreaker clears the circuit breaker so the next Send dials cloud again.\n// Called when a cloud status check succeeds, proving the API key is valid.\nfunc (c *CloudDispatcher) ResetBreaker() {\n\tc.blockedUntil.Store(0)\n}\n\n// Send sends a notification to Dozzle Cloud\nfunc (c *CloudDispatcher) Send(ctx context.Context, notification types.Notification) error {\n\tif blockedUntil := c.blockedUntil.Load(); blockedUntil > 0 && time.Now().UnixNano() < blockedUntil {\n\t\tt := time.Unix(0, blockedUntil)\n\t\tlog.Debug().\n\t\t\tStr(\"cloud\", c.Name).\n\t\t\tTime(\"blocked_until\", t).\n\t\t\tMsg(\"circuit breaker open, skipping cloud request\")\n\t\treturn fmt.Errorf(\"cloud dispatcher rate limited, retry after %s\", t.Format(time.RFC3339))\n\t}\n\n\tpayload, err := json.Marshal(notification)\n\tif err != nil {\n\t\treturn fmt.Errorf(\"failed to marshal notification: %w\", err)\n\t}\n\n\treq, err := http.NewRequestWithContext(ctx, http.MethodPost, c.URL, bytes.NewReader(payload))\n\tif err != nil {\n\t\treturn fmt.Errorf(\"failed to create request: %w\", err)\n\t}\n\n\treq.Header.Set(\"Content-Type\", \"application/json\")\n\treq.Header.Set(\"User-Agent\", UserAgent)\n\treq.Header.Set(\"X-API-Key\", c.APIKey)\n\n\tresp, err := c.client.Do(req)\n\tif err != nil {","sourceCodeStart":57,"sourceCodeEnd":93,"githubUrl":"https://github.com/amir20/dozzle/blob/d9463cbe21874e44ab79db6fa63e746ca7d22928/internal/notification/dispatcher/cloud.go#L57-L93","documentation":"CloudDispatcher.Send checks an internal circuit breaker (blockedUntil atomic timestamp) before dialing Dozzle Cloud. When a previous 429 or auth rejection tripped the breaker and now is still inside the backoff window, Send skips the HTTP request entirely and returns this error carrying the RFC3339 time until which all sends are blocked. It is a deliberate client-side rate limiter, not a remote failure at send time.","triggerScenarios":"Calling CloudDispatcher.Send after an earlier send received HTTP 429 (backoff set, default 60s or Retry-After) or 401/403 (backoff set to 6h), while time.Now() is still before blockedUntil.","commonSituations":"Frequent alert bursts hitting Dozzle Cloud rate limits; an expired or invalid DOLIGENCE API key that tripped the 6h breaker on the first 401/403; notification rules firing in a loop.","solutions":["Wait until the retry-after timestamp in the error message before sending again","Fix the root cause: verify the API key is valid; if a status check succeeds, call ResetBreaker() to clear blockedUntil","Reduce notification volume or debounce rules so 429s stop re-tripping the breaker","Recreate the dispatcher with a fresh valid API key, which resets the breaker state"],"exampleFix":"// before\nerr := dispatcher.Send(ctx, n) // fails while breaker is open\n// after\nif err := dispatcher.Send(ctx, n); err != nil && strings.Contains(err.Error(), \"retry after\") {\n    // parse timestamp, retry after that time, or check key validity and call ResetBreaker()\n}","handlingStrategy":"retry","validationCode":null,"typeGuard":null,"tryCatchPattern":"if err := d.Send(ctx, n); err != nil {\n    if strings.HasPrefix(err.Error(), \"cloud dispatcher rate limited\") {\n        // parse RFC3339 timestamp from message; schedule retry after it\n    }\n}","preventionTips":["Check breaker state before bursting sends; back off on 429 responses","Fix invalid API keys promptly; call ResetBreaker() after a successful status check","Debounce high-frequency notification rules"],"tags":["rate-limit","circuit-breaker","http","notifications"],"backgroundTag":"rate-limit-exceeded","analyzedSha":"d9463cbe21874e44ab79db6fa63e746ca7d22928","analyzedAt":"2026-09-07T10:08:55.855Z","contentChangedAt":"2026-09-07T10:08:55.855Z","schemaVersion":2},"datasetVersion":"2026-09-16T04:17:20.429Z"}