{"record":{"id":"60f4ef02663a635e","repo":"Tencent/WeKnora","slug":"download-file-w","errorCode":null,"errorMessage":"download file: %w","messagePattern":"download file: %w","errorType":"exception","errorClass":null,"httpStatus":null,"severity":"error","filePath":"internal/datasource/connector/notion/client.go","lineNumber":427,"sourceCode":"\t\t\t\t\treturn nil, sErr\n\t\t\t\t}\n\t\t\t\tcontinue\n\t\t\t}\n\t\t\tbreak\n\t\t}\n\n\t\tdata, err := io.ReadAll(io.LimitReader(resp.Body, maxDownloadSize+1))\n\t\tresp.Body.Close()\n\t\tif err != nil {\n\t\t\treturn nil, err\n\t\t}\n\t\tif int64(len(data)) > maxDownloadSize {\n\t\t\treturn nil, fmt.Errorf(\"file exceeds maximum download size (%d MB)\", maxDownloadSize/(1024*1024))\n\t\t}\n\t\treturn data, nil\n\t}\n\n\treturn nil, fmt.Errorf(\"download file: %w\", lastErr)\n}\n\n// --- Shared pagination helper ---\n\n// paginatePages fetches all pages from a paginated Notion API endpoint.\nfunc (c *notionClient) paginatePages(ctx context.Context, method, path string) ([]notionPage, error) {\n\tvar allPages []notionPage\n\tvar startCursor string\n\n\tfor {\n\t\tbody := map[string]interface{}{\n\t\t\t\"page_size\": 100,\n\t\t}\n\t\tif startCursor != \"\" {\n\t\t\tbody[\"start_cursor\"] = startCursor\n\t\t}\n\n\t\tvar respBody []byte","sourceCodeStart":409,"sourceCodeEnd":445,"githubUrl":"https://github.com/Tencent/WeKnora/blob/988cbb03305e055d8ebb7d46d9ac6cc0803cd074/internal/datasource/connector/notion/client.go#L409-L445","documentation":"DownloadFile retries HTTP requests up to maxRetries times. If every attempt fails (network error or exhausted retries on non-200 status), the last error is wrapped as \"download file: %w\" and returned. It is a generic transport-level failure for attachment downloads from Notion's file hosts.","triggerScenarios":"All retry attempts in DownloadFile fail: transient network errors, DNS failures, or repeated non-200 responses (403/404/500) from the attachment URL after retries are exhausted.","commonSituations":"Expired or revoked Notion file URLs (signed S3 URLs expire), outage of Notion's file CDN, connector host has no outbound internet access, attachment deleted between listing and download.","solutions":["Inspect the wrapped lastErr (via errors.Unwrap or %v) to see the actual cause","Re-run the fetch; signed attachment URLs are transient, a fresh page fetch yields new URLs","Check outbound network/proxy access from the connector host","Re-index the page so Notion issues a fresh download URL"],"exampleFix":"data, err := client.DownloadFile(ctx, fileURL)\nif err != nil {\n    return fmt.Errorf(\"skipping attachment: %w\", err) // don't fail whole page\n}","handlingStrategy":"retry","validationCode":"u, err := url.Parse(fileURL); if err != nil || u.Scheme != \"https\" { return err }","typeGuard":null,"tryCatchPattern":"data, err := client.DownloadFile(ctx, url)\nif err != nil {\n    var urlErr *url.Error\n    if errors.As(err, &urlErr) && isTransient(urlErr) {\n        // schedule re-fetch later; signed URLs expire\n    }\n    return fmt.Errorf(\"attachment download skipped: %w\", err)\n}","preventionTips":["Treat attachment URLs as ephemeral — re-fetch pages to get fresh URLs","Add backoff retry at the caller level for transient network errors","Monitor outbound connectivity from the connector host"],"tags":["notion","http","network","download"],"backgroundTag":"http-request-failed","analyzedSha":"988cbb03305e055d8ebb7d46d9ac6cc0803cd074","analyzedAt":"2026-09-02T14:41:08.344Z","contentChangedAt":null,"schemaVersion":2},"datasetVersion":"2026-09-08T10:18:20.063Z"}