multica-ai/multica · warning · ErrCloudPATUnavailable

cloud pat verifier unavailable

Error message

cloud pat verifier unavailable

What it means

Sentinel error ErrCloudPATUnavailable in server/internal/auth/cloud_pat.go: the server could not verify an mcn_ PAT because the Fleet verification endpoint was unreachable, timed out (5s default), or returned a 5xx. Middlewares map it to HTTP 503 — the token may be perfectly valid; verification infrastructure is what failed. Callers must not treat it as an auth rejection.

Source

Thrown at server/internal/auth/cloud_pat.go:83

// cloudPATDefaultTimeout is the per-request HTTP timeout for verify
// calls when the caller doesn't supply an *http.Client. Auth must
// stay snappy: Fleet should answer in tens of milliseconds, and a
// hung verify would block every incoming request behind it. Tighter
// than cloudruntime's 35s because that one proxies arbitrary user
// traffic; this one only ever sees a small JSON exchange.
const cloudPATDefaultTimeout = 5 * time.Second

// Verifier sentinel errors. Callers (the Auth / DaemonAuth middlewares)
// branch on these to map cloud outcomes onto HTTP status codes:
//
//   - ErrCloudPATInvalid       → 401 (Fleet says token is bad)
//   - ErrCloudPATUnavailable   → 503 (Fleet unreachable / 5xx)
//   - ErrCloudPATNotConfigured → 401 (server has no Fleet URL set; we
//     don't reveal that mcn_ is "supported but disabled" — failing
//     closed avoids treating misconfigured prod the same as enabled)
var (
	ErrCloudPATInvalid       = errors.New("cloud pat invalid")
	ErrCloudPATUnavailable   = errors.New("cloud pat verifier unavailable")
	ErrCloudPATNotConfigured = errors.New("cloud pat verifier not configured")
)

// CloudPATIdentity is what a successful verify resolves to. We keep
// only the fields the auth path actually needs:
//
//   - OwnerID is the user whose request this is (mapped to X-User-ID).
//   - InstanceID / InstanceRecordID are recorded so downstream code can
//     correlate the request with a specific cloud node; they are not
//     used for authorization today, but stashing them now keeps the
//     wire shape stable for callers that later want to assert a
//     particular instance binding.
//
// We deliberately drop token_last4, status, issued_at, etc. — those
// are diagnostic fields that don't belong in cached auth state.
type CloudPATIdentity struct {
	OwnerID          string `json:"o"`
	InstanceID       string `json:"i"`

View on GitHub (pinned to 2c0912b6ec)

Solutions

  1. Retry after a short delay — transient Fleet unavailability resolves itself; the 503 signals retryability.
  2. Check Fleet status/health endpoint and the network path from the Multica server host (egress rules, proxy, DNS).
  3. If it persists, verify the configured Fleet URL and any proxy env vars on the server.
  4. Never invalidate or rotate the PAT in response to this error — it is not a rejection.

Example fix

// go — caller-side branching
if errors.Is(err, auth.ErrCloudPATUnavailable) {
    http.Error(w, "cloud auth temporarily unavailable, retry", http.StatusServiceUnavailable)
    return
}
Defensive patterns

Strategy: retry

Try / catch

if errors.Is(err, auth.ErrCloudPATUnavailable) {
    // 503 + Retry-After; do NOT treat as auth rejection or clear the token
    w.Header().Set("Retry-After", "5")
    http.Error(w, "cloud auth temporarily unavailable", http.StatusServiceUnavailable)
    return
}

Prevention

When it happens

Trigger: Fleet service down or restarting when the request arrived; network partition/DNS failure between the Multica server and Fleet; Fleet returning 500s; verification exceeding the 5s cloudPATDefaultTimeout.

Common situations: Cloud outage windows; egress firewall blocking the Fleet URL from the server host; DNS misconfiguration; Fleet deploy in progress.

Related errors

AI-assisted analysis of multica-ai/multica@2c0912b6ec (2026-08-15). Data as JSON: /api/errors/0ec3745afc39ee7a. Report an issue: GitHub.