{"record":{"id":"019a1deb3a8c9bcf","repo":"toeverything/AFFiNE","slug":"too-many-request-019a1d","errorCode":"too_many_request","errorMessage":"Too many requests.","messagePattern":"Too many requests\\.","errorType":"exception","errorClass":"TooManyRequest","httpStatus":429,"severity":"error","filePath":"packages/backend/server/src/core/workspaces/abuse.ts","lineNumber":290,"sourceCode":"          ? 'fail_open'\n          : 'fail_closed',\n      });\n      this.logger.error('Workspace invite quota native assert failed', {\n        userId: input.actorUserId,\n        workspaceId: input.workspaceId,\n        targetCount: input.targetCount,\n        targetDomainsSummary: input.targetDomains,\n        sourceTrusted: input.source?.trusted ?? false,\n        country: input.source?.country,\n        asn: input.source?.asn,\n        requestId: input.requestId,\n        cfRay: input.source?.rayId,\n        error: message,\n      });\n      if (this.config.auth.inviteQuotaFailOpenOnRuntimeError) {\n        return { decision: { allowed: true, requested: input.targetCount } };\n      }\n      throw new TooManyRequest();\n    }\n    metrics.workspace\n      .counter('invite_quota_requested_targets')\n      .add(input.targetCount);\n    metrics.workspace\n      .histogram('invite_quota_counter_latency_ms')\n      .record(Date.now() - start);\n\n    if (!decision.allowed) {\n      metrics.workspace.counter('invite_quota_rejected').add(1, {\n        reason: decision.reason ?? 'unknown',\n      });\n      metrics.workspace.counter('invite_quota_reject_by_reason').add(1, {\n        reason: decision.reason ?? 'unknown',\n      });\n      if (this.config.auth.inviteQuotaShadowMode) {\n        this.logger.warn('Workspace invite quota shadow rejected', {\n          userId: input.actorUserId,","sourceCodeStart":272,"sourceCodeEnd":308,"githubUrl":"https://github.com/toeverything/AFFiNE/blob/b6de0ad51b76f3daac2d3d6325369ea623ed7ed4/packages/backend/server/src/core/workspaces/abuse.ts#L272-L308","documentation":"Thrown by InviteQuotaAssertService.assertWorkspaceInviteQuota when the external invite-quota runtime (runtime.assertWorkspaceInviteQuotaV1) throws. The catch block is fail-closed by default: unless config.auth.inviteQuotaFailOpenOnRuntimeError is true, the server rethrows TooManyRequest (HTTP 429, code too_many_request) rather than let a broken limiter silently admit invite spam. A genuine quota rejection on the allowed path also surfaces under the same code, so the server log line 'Workspace invite quota native assert failed' is what distinguishes this infra-failure variant.","triggerScenarios":"Calling workspace invite mutations while the quota runtime errors: connectivity failure to the abuse-control backend, missing/misconfigured runtime credentials, or a malformed decision payload — with failOpen disabled, every such runtime failure becomes 429 for the inviter.","commonSituations":"Self-hosted deployments that enabled the invite abuse runtime without provisioning its backend; transient network partitions between app server and quota service; cloud config copied to an environment lacking the cloud infrastructure; misreading this 429 as user-level rate limiting when it is actually fail-closed infra.","solutions":["Grep server logs for 'Workspace invite quota native assert failed' — its error: field names the real runtime failure (auth, DNS, timeout)","Fix or provision the quota runtime backend so assertWorkspaceInviteQuotaV1 succeeds","Self-host: set auth.inviteQuotaFailOpenOnRuntimeError=true so an unavailable limiter degrades to allow instead of 429","Retry the invite once the underlying issue is resolved; watch the invite_quota_runtime_fallback metric to confirm the mode"],"exampleFix":null,"handlingStrategy":"retry","validationCode":null,"typeGuard":null,"tryCatchPattern":"let lastErr;\nfor (const delay of [1000, 5000, 15000]) {\n  try {\n    return await inviteMembers(wsId, emails);\n  } catch (e) {\n    if (e?.code === 'too_many_request' && serverLogsShowRuntimeFallback()) {\n      lastErr = e;\n      await sleep(delay); // infra fail-closed, not user abuse — safe to retry\n      continue;\n    }\n    throw e;\n  }\n}\nthrow lastErr;","preventionTips":["Monitor the invite_quota_runtime_fallback metric; sustained fail_closed mode means the quota backend is down","Self-host: set auth.inviteQuotaFailOpenOnRuntimeError=true if you accept open invites when the limiter is unavailable","Provision and health-check the invite-quota runtime backend before enabling the feature","Surface a distinct UI message for this 429 ('invite service unavailable, retrying') instead of 'you are rate limited'"],"tags":["rate-limit","invite","quota","fail-closed","infrastructure"],"backgroundTag":"rate-limit-exceeded","analyzedSha":"b6de0ad51b76f3daac2d3d6325369ea623ed7ed4","analyzedAt":"2026-08-18T21:16:52.546Z","contentChangedAt":"2026-08-18T21:16:52.546Z","schemaVersion":2},"datasetVersion":"2026-09-23T08:17:48.524Z"}