{"record":{"id":"f1db860c51f45d0e","repo":"multica-ai/multica","slug":"cloud-pat-verifier-unavailable-f1db86","errorCode":null,"errorMessage":"cloud pat verifier unavailable","messagePattern":"cloud pat verifier unavailable","errorType":"http","errorClass":null,"httpStatus":503,"severity":"warning","filePath":"server/internal/middleware/auth.go","lineNumber":136,"sourceCode":"\t\t\t// pass with a phantom X-User-ID.\n\t\t\tif strings.HasPrefix(tokenString, auth.CloudPATPrefix) {\n\t\t\t\tif cloudPAT == nil {\n\t\t\t\t\tslog.Warn(\"auth: mcn_ token presented but cloud verifier not configured\", \"path\", r.URL.Path)\n\t\t\t\t\thttp.Error(w, `{\"error\":\"invalid token\"}`, http.StatusUnauthorized)\n\t\t\t\t\treturn\n\t\t\t\t}\n\t\t\t\tidentity, err := cloudPAT.Verify(r.Context(), tokenString, ownerLookupFor(queries))\n\t\t\t\tif err != nil {\n\t\t\t\t\tif errors.Is(err, auth.ErrCloudPATInvalid) {\n\t\t\t\t\t\tslog.Warn(\"auth: cloud rejected mcn_ token\", \"path\", r.URL.Path, \"error\", err)\n\t\t\t\t\t\thttp.Error(w, `{\"error\":\"invalid token\"}`, http.StatusUnauthorized)\n\t\t\t\t\t\treturn\n\t\t\t\t\t}\n\t\t\t\t\t// Cloud unreachable / 5xx / decode error. We surface\n\t\t\t\t\t// 503 so callers (CLI / daemon) can retry — a 401\n\t\t\t\t\t// here would tell them to throw out a valid token.\n\t\t\t\t\tslog.Warn(\"auth: cloud pat verify unavailable\", \"path\", r.URL.Path, \"error\", err)\n\t\t\t\t\thttp.Error(w, `{\"error\":\"cloud pat verifier unavailable\"}`, http.StatusServiceUnavailable)\n\t\t\t\t\treturn\n\t\t\t\t}\n\t\t\t\tr.Header.Set(\"X-User-ID\", identity.OwnerID)\n\t\t\t\t// Tag the auth path so account-level guards (e.g.\n\t\t\t\t// handler.RequireHumanActor on /api/cloud-billing/*)\n\t\t\t\t// can distinguish a cloud-node machine credential\n\t\t\t\t// from a human PAT/JWT. Mirrors the mat_ branch's\n\t\t\t\t// stamp of \"task_token\" — both are server-set,\n\t\t\t\t// authoritative, and stripped from any client-\n\t\t\t\t// supplied value at the top of this middleware. Same\n\t\t\t\t// rationale as MUL-2600: a machine credential\n\t\t\t\t// (running agent or running cloud node) must not be\n\t\t\t\t// treated as the owner having approved an account-\n\t\t\t\t// level action.\n\t\t\t\tr.Header.Set(\"X-Actor-Source\", \"cloud_pat\")\n\t\t\t\tnext.ServeHTTP(w, r)\n\t\t\t\treturn\n\t\t\t}","sourceCodeStart":118,"sourceCodeEnd":154,"githubUrl":"https://github.com/multica-ai/multica/blob/2c0912b6ec764b373d44eeea1e80f0d9f11ab417/server/internal/middleware/auth.go#L118-L154","documentation":"HTTP 503 returned when cloud PAT verification could not be completed because Cloud was unreachable, returned a 5xx, or the response failed to decode — anything other than an authoritative 'invalid' verdict. The middleware deliberately uses 503 rather than 401 so CLI/daemon callers retry instead of discarding a possibly valid token; the distinction is logged as 'cloud pat verify unavailable' at Warn level.","triggerScenarios":"Cloud verify endpoint is down or timing out; network egress blocked from the server; TLS failure to the cloud host; malformed/unexpected response body from Cloud (version skew).","commonSituations":"Cloud outage; firewall/DNS problems on self-hosted servers; transient 5xx during cloud deploys; proxy misconfiguration intercepting cloud traffic.","solutions":["Retry the request after a backoff — the token is not invalidated by this response.","Check server → cloud connectivity (curl the cloud verify endpoint from the server host).","Inspect firewall/DNS/proxy config for the cloud host on the server.","If persistent, compare server and cloud versions for response-format skew and check cloud status page."],"exampleFix":"// before: treat any non-2xx as bad token and log out\nif resp.StatusCode != 200 { logout() }\n\n// after: 503 means retry, 401 means re-auth\nif resp.StatusCode == http.StatusServiceUnavailable {\n    time.Sleep(backoff.Next()); retry(req)\n} else if resp.StatusCode == http.StatusUnauthorized {\n    refreshToken()\n}","handlingStrategy":"retry","validationCode":"// preflight cloud reachability before a batch of authenticated calls\nif err := pingCloudVerify(ctx, 2*time.Second); err != nil {\n    return fmt.Errorf(\"cloud verifier unreachable, delaying batch: %w\", err)\n}","typeGuard":null,"tryCatchPattern":"var bo backoff.ExponentialBackOff\nfor attempt := 0; attempt < 5; attempt++ {\n    resp, err := client.Do(req)\n    if err == nil && resp.StatusCode != http.StatusServiceUnavailable { break }\n    time.Sleep(bo.NextAttempt())\n}","preventionTips":["Never treat 503 from this path as token invalidation — keep the credential.","Add egress health checks to the cloud host in server monitoring.","Use jittered exponential backoff to ride out cloud deploys."],"tags":["cloud","network","retry","http-503","availability"],"backgroundTag":null,"analyzedSha":"2c0912b6ec764b373d44eeea1e80f0d9f11ab417","analyzedAt":"2026-08-15T13:25:18.241Z","schemaVersion":2},"datasetVersion":"2026-08-15T22:17:37.221Z"}