{"record":{"id":"c39a72d3af79823b","repo":"Tencent/WeKnora","slug":"rate-limited-s","errorCode":null,"errorMessage":"rate limited: %s","messagePattern":"rate limited: (.+?)","errorType":"http","errorClass":null,"httpStatus":429,"severity":"warning","filePath":"internal/datasource/connector/notion/client.go","lineNumber":126,"sourceCode":"\n\t\tswitch {\n\t\tcase resp.StatusCode >= 200 && resp.StatusCode < 300:\n\t\t\treturn respBody, nil\n\n\t\tcase resp.StatusCode == 401 || resp.StatusCode == 403:\n\t\t\treturn nil, fmt.Errorf(\"%w: %s\", datasource.ErrInvalidCredentials, string(respBody))\n\n\t\tcase resp.StatusCode == 404:\n\t\t\treturn nil, fmt.Errorf(\"%w: %s\", datasource.ErrResourceNotFound, path)\n\n\t\tcase resp.StatusCode == 429:\n\t\t\tretryAfter := resp.Header.Get(\"Retry-After\")\n\t\t\twait := 1 * time.Second\n\t\t\tif secs, err := strconv.ParseFloat(retryAfter, 64); err == nil && secs > 0 {\n\t\t\t\twait = time.Duration(secs * float64(time.Second))\n\t\t\t}\n\t\t\tlogger.Warnf(ctx, \"[Notion] rate limited, retry after %v (attempt %d/%d)\", wait, attempt+1, maxRetries)\n\t\t\tlastErr = fmt.Errorf(\"rate limited: %s\", string(respBody))\n\t\t\tif attempt < maxRetries {\n\t\t\t\tif sErr := sleepWithContext(ctx, wait); sErr != nil {\n\t\t\t\t\treturn nil, sErr\n\t\t\t\t}\n\t\t\t\tcontinue\n\t\t\t}\n\n\t\tcase resp.StatusCode >= 500:\n\t\t\tlastErr = fmt.Errorf(\"server error %d: %s\", resp.StatusCode, string(respBody))\n\t\t\tif attempt < maxRetries {\n\t\t\t\tif sErr := sleepWithContext(ctx, time.Duration(1<<attempt)*time.Second); sErr != nil {\n\t\t\t\t\treturn nil, sErr\n\t\t\t\t}\n\t\t\t\tcontinue\n\t\t\t}\n\n\t\tdefault:\n\t\t\treturn nil, fmt.Errorf(\"unexpected status %d: %s\", resp.StatusCode, string(respBody))","sourceCodeStart":108,"sourceCodeEnd":144,"githubUrl":"https://github.com/Tencent/WeKnora/blob/988cbb03305e055d8ebb7d46d9ac6cc0803cd074/internal/datasource/connector/notion/client.go#L108-L144","documentation":"The Notion API returned HTTP 429 (Too Many Requests). The client honors the Retry-After header (defaulting to 1s) and retries up to maxRetries=3 times. This error only escapes when all retry attempts are exhausted; it is then wrapped as `fmt.Errorf(\"%w: %v\", datasource.ErrFetchFailed, lastErr)` before returning, so callers see the wrapped message. The body of the 429 response is embedded in the message.","triggerScenarios":"Any doRequest call (Ping, GetPage, GetDatabaseInfo, GetDataSourceInfo, GetBlockChildrenFlat, getBlockChildrenRecursive) that receives 429 on attempt 0, 1, 2, AND 3 — i.e. Notion keeps rate limiting for longer than the Retry-After durations (~1s each, plus the 3 req/s local limiter).","commonSituations":"Bulk initial sync of a large workspace where many pages/databases are fetched back-to-back; multiple WeKnora workers sharing the same Notion integration token (limits are per-token per-workspace); Retry-After headers of several seconds or minutes exceeding the retry budget.","solutions":["Wait and retry the sync later — the error means the token's Notion rate budget is exhausted for now.","Reduce concurrency: run one sync at a time per integration token, and avoid sharing one token across multiple jobs or services.","Lower the client's rate limit (rate.NewLimiter(rate.Limit(3), 3) in newClient, client.go:40) to leave headroom under Notion's ~3 req/s limit.","Increase maxRetries (client.go:45) if sustained bursts are expected, so long Retry-After waits are honored.","For repeatedly-hit limits on large workspaces, paginate with smaller page_size and add backoff between pages in the caller."],"exampleFix":"// before: shared token across parallel workers hitting 429\n// after: single-flight sync + reduced limiter\nlimiter: rate.NewLimiter(rate.Limit(2), 2), // was rate.Limit(3), 3","handlingStrategy":"retry","validationCode":"// Check client-side budget before starting a large sync\nfunc canSync(limiter *rate.Limiter) bool { return limiter.Allow() }","typeGuard":null,"tryCatchPattern":"if err := client.Ping(ctx); err != nil {\n    var fetchErr *datasource.FetchError\n    if strings.Contains(err.Error(), \"rate limited\") {\n        time.Sleep(30 * time.Second)\n        // re-enqueue the job instead of failing the sync\n    }\n    _ = fetchErr\n}","preventionTips":["One sync job at a time per Notion integration token","Keep the client limiter (3 req/s) at or below Notion's documented limit","Avoid sharing one integration token across multiple services/environments","Schedule large workspace initial syncs off-peak"],"tags":["notion","rate-limit","http-429","retry","api"],"backgroundTag":"api-rate-limit-exceeded","analyzedSha":"988cbb03305e055d8ebb7d46d9ac6cc0803cd074","analyzedAt":"2026-09-02T14:41:08.344Z","contentChangedAt":null,"schemaVersion":2},"datasetVersion":"2026-09-08T10:18:20.063Z"}