{"record":{"id":"96a8f80907397d1a","repo":"Tencent/WeKnora","slug":"file-exceeds-maximum-download-size-d-mb","errorCode":null,"errorMessage":"file exceeds maximum download size (%d MB)","messagePattern":"file exceeds maximum download size \\((.+?) MB\\)","errorType":"exception","errorClass":null,"httpStatus":null,"severity":"warning","filePath":"internal/datasource/connector/notion/client.go","lineNumber":422,"sourceCode":"\t\tif resp.StatusCode != http.StatusOK {\n\t\t\tresp.Body.Close()\n\t\t\tlastErr = fmt.Errorf(\"download failed with status %d\", resp.StatusCode)\n\t\t\tif resp.StatusCode >= 500 && attempt < maxRetries {\n\t\t\t\tif sErr := sleepWithContext(ctx, time.Duration(1<<attempt)*time.Second); sErr != nil {\n\t\t\t\t\treturn nil, sErr\n\t\t\t\t}\n\t\t\t\tcontinue\n\t\t\t}\n\t\t\tbreak\n\t\t}\n\n\t\tdata, err := io.ReadAll(io.LimitReader(resp.Body, maxDownloadSize+1))\n\t\tresp.Body.Close()\n\t\tif err != nil {\n\t\t\treturn nil, err\n\t\t}\n\t\tif int64(len(data)) > maxDownloadSize {\n\t\t\treturn nil, fmt.Errorf(\"file exceeds maximum download size (%d MB)\", maxDownloadSize/(1024*1024))\n\t\t}\n\t\treturn data, nil\n\t}\n\n\treturn nil, fmt.Errorf(\"download file: %w\", lastErr)\n}\n\n// --- Shared pagination helper ---\n\n// paginatePages fetches all pages from a paginated Notion API endpoint.\nfunc (c *notionClient) paginatePages(ctx context.Context, method, path string) ([]notionPage, error) {\n\tvar allPages []notionPage\n\tvar startCursor string\n\n\tfor {\n\t\tbody := map[string]interface{}{\n\t\t\t\"page_size\": 100,\n\t\t}","sourceCodeStart":404,"sourceCodeEnd":440,"githubUrl":"https://github.com/Tencent/WeKnora/blob/988cbb03305e055d8ebb7d46d9ac6cc0803cd074/internal/datasource/connector/notion/client.go#L404-L440","documentation":"DownloadFile in the Notion connector enforces a hard cap (maxDownloadSize) on attachment downloads. It reads at most maxDownloadSize+1 bytes with io.LimitReader; if the extra byte is present the file is over the limit and the download is aborted with this error. This protects memory from oversized Notion-hosted file attachments.","triggerScenarios":"Calling DownloadFile (via fetchPage while indexing) on an attachment whose content is larger than maxDownloadSize bytes, e.g. a large video, ZIP archive, or high-resolution image hosted by Notion.","commonSituations":"Notion workspaces that store large PDFs/videos/exports as page attachments; indexing a database containing file-type properties pointing at big uploads.","solutions":["Skip or exclude the oversized attachment from indexing instead of downloading it whole","Stream the file to disk instead of reading it fully into memory if it must be stored","Raise the connector's maxDownloadSize limit if your deployment has the memory budget"],"exampleFix":"// before\ndata, err := client.DownloadFile(ctx, fileURL)\n// after\nif isOversizedAttachment(fileSizeBytes) { // check metadata/size first\n    log.Printf(\"skipping attachment %s: too large\", fileURL)\n    return nil\n}\ndata, err := client.DownloadFile(ctx, fileURL)","handlingStrategy":"validation","validationCode":"if fileSizeKnown && int64(fileSizeKnown) > maxDownloadSize { skipAttachment(fileURL); return }","typeGuard":"func isWithinLimit(n int64, max int64) bool { return n > 0 && n <= max }","tryCatchPattern":"data, err := client.DownloadFile(ctx, url)\nif err != nil {\n    if strings.Contains(err.Error(), \"file exceeds maximum download size\") {\n        return nil // skip large files gracefully\n    }\n    return err\n}","preventionTips":["Check Content-Length or Notion file metadata size before downloading","Exclude known-large file types (video, archives) from indexing","Stream large files to disk instead of buffering if full download is required"],"tags":["notion","file-size-limit","attachment-download"],"backgroundTag":"file-size-limit-exceeded","analyzedSha":"988cbb03305e055d8ebb7d46d9ac6cc0803cd074","analyzedAt":"2026-09-02T14:41:08.344Z","contentChangedAt":null,"schemaVersion":2},"datasetVersion":"2026-09-08T10:18:20.063Z"}