{"record":{"id":"7e51afbf8de1e887","repo":"Tencent/WeKnora","slug":"download-failed-with-status-d","errorCode":null,"errorMessage":"download failed with status %d","messagePattern":"download failed with status (.+?)","errorType":"http","errorClass":null,"httpStatus":null,"severity":"error","filePath":"internal/datasource/connector/notion/client.go","lineNumber":406,"sourceCode":"\t}\n\n\tvar lastErr error\n\tfor attempt := 0; attempt <= maxRetries; attempt++ {\n\t\tresp, err := c.httpClient.Do(req)\n\t\tif err != nil {\n\t\t\tlastErr = err\n\t\t\tif attempt < maxRetries {\n\t\t\t\tif sErr := sleepWithContext(ctx, time.Duration(1<<attempt)*time.Second); sErr != nil {\n\t\t\t\t\treturn nil, sErr\n\t\t\t\t}\n\t\t\t\tcontinue\n\t\t\t}\n\t\t\tbreak\n\t\t}\n\n\t\tif resp.StatusCode != http.StatusOK {\n\t\t\tresp.Body.Close()\n\t\t\tlastErr = fmt.Errorf(\"download failed with status %d\", resp.StatusCode)\n\t\t\tif resp.StatusCode >= 500 && attempt < maxRetries {\n\t\t\t\tif sErr := sleepWithContext(ctx, time.Duration(1<<attempt)*time.Second); sErr != nil {\n\t\t\t\t\treturn nil, sErr\n\t\t\t\t}\n\t\t\t\tcontinue\n\t\t\t}\n\t\t\tbreak\n\t\t}\n\n\t\tdata, err := io.ReadAll(io.LimitReader(resp.Body, maxDownloadSize+1))\n\t\tresp.Body.Close()\n\t\tif err != nil {\n\t\t\treturn nil, err\n\t\t}\n\t\tif int64(len(data)) > maxDownloadSize {\n\t\t\treturn nil, fmt.Errorf(\"file exceeds maximum download size (%d MB)\", maxDownloadSize/(1024*1024))\n\t\t}\n\t\treturn data, nil","sourceCodeStart":388,"sourceCodeEnd":424,"githubUrl":"https://github.com/Tencent/WeKnora/blob/988cbb03305e055d8ebb7d46d9ac6cc0803cd074/internal/datasource/connector/notion/client.go#L388-L424","documentation":"DownloadFile treats any non-200 response from the attachment host as a failure and records 'download failed with status %d'. 5xx statuses are retried up to maxRetries with exponential backoff (1s, 2s, ...); 4xx and persistent 5xx statuses break out and the error surfaces wrapped as 'download file: download failed with status N'. S3 signed URLs are the usual target, so 403 (expired/invalid signature) and 404 (deleted file) are the dominant cases.","triggerScenarios":"The signed S3 URL has expired (Notion file URLs expire after ~1 hour) yielding 403; the attachment was deleted from the page yielding 404; the storage endpoint returns 500/503 beyond the retry budget; a rate-limited 429 from the storage host.","commonSituations":"Long-running syncs that hold block data for over an hour before downloading; retrying a job hours after fetching the URL list; attachments removed between indexing and download; transient storage outages exceeding maxRetries.","solutions":["For 403, re-fetch the block via ResolveBlock to obtain a fresh signed URL, then retry the download immediately.","For 404, check whether the attachment still exists on the page in Notion; remove it from the index or flag the record as stale.","For 5xx after retries, re-run the sync later — the retry loop already applies exponential backoff, so adding retries in the caller rarely helps.","Shorten the gap between URL retrieval and download so signed URLs are used within their 1-hour validity window."],"exampleFix":"// before\nblock, _ := client.ResolveBlock(ctx, blockID) // fetched hours earlier, signed URL expired\n// after\nblock, _ := client.ResolveBlock(ctx, blockID) // re-resolve right before download\ndata, err := client.DownloadFile(ctx, block.File.File.URL) // fresh 1-hour signed URL","handlingStrategy":"retry","validationCode":"// signed URLs expire after ~1 hour; check age before downloading\nif time.Since(block.FetchedAt) > 45*time.Minute {\n    block, err = client.ResolveBlock(ctx, block.ID)\n    if err != nil {\n        return err\n    }\n}","typeGuard":"func isDownloadStatusError(err error, statuses ...int) bool {\n    if err == nil {\n        return false\n    }\n    for _, s := range statuses {\n        if strings.Contains(err.Error(), fmt.Sprintf(\"download failed with status %d\", s)) {\n            return true\n        }\n    }\n    return false\n}","tryCatchPattern":"data, err := client.DownloadFile(ctx, fileURL)\nif err != nil {\n    if isDownloadStatusError(err, 403) {\n        // signed URL expired — re-resolve and retry once\n        if fresh, rErr := client.ResolveBlock(ctx, blockID); rErr == nil {\n            return client.DownloadFile(ctx, fresh.File.File.URL)\n        }\n    }\n    if isDownloadStatusError(err, 404) {\n        return nil // attachment deleted; skip\n    }\n    return err // 5xx already retried with backoff inside DownloadFile\n}","preventionTips":["Download attachments immediately after resolving blocks — signed URLs expire in about an hour.","Handle 403 as 're-resolve URL' and 404 as 'attachment gone' rather than generic retries.","Avoid scheduling downloads of long-stored URL lists; persist block IDs and re-resolve instead."],"tags":["http","download","s3","notion","retry"],"backgroundTag":"http-403-forbidden","analyzedSha":"988cbb03305e055d8ebb7d46d9ac6cc0803cd074","analyzedAt":"2026-09-02T14:41:08.344Z","contentChangedAt":null,"schemaVersion":2},"datasetVersion":"2026-09-08T10:18:20.063Z"}