{"record":{"id":"44d27d6d25a2c03a","repo":"Tencent/WeKnora","slug":"download-zip-w","errorCode":null,"errorMessage":"download zip: %w","messagePattern":"download zip: %w","errorType":"exception","errorClass":null,"httpStatus":null,"severity":"error","filePath":"internal/infrastructure/docparser/mineru_cloud_converter.go","lineNumber":362,"sourceCode":"\t\treturn \"\", nil, fmt.Errorf(\"extract zip: %w\", err)\n\t}\n\n\tlogger.Infof(context.Background(), \"[MinerUCloud] parsed (zip), markdown=%d chars, images=%d\", len(md), len(imageRefs))\n\treturn md, imageRefs, nil\n}\n\n// --- ZIP handling ---\n\nvar imgRefPattern = regexp.MustCompile(`!\\[[^\\]]*\\]\\(([^)]+)\\)`)\n\nfunc downloadAndExtractZip(zipURL string) (string, []types.ImageRef, error) {\n\tif err := utils.ValidateURLForSSRF(zipURL); err != nil {\n\t\treturn \"\", nil, fmt.Errorf(\"zip URL blocked by SSRF check: %v\", err)\n\t}\n\tclient := utils.NewSSRFSafeHTTPClient(utils.SSRFSafeHTTPClientConfig{Timeout: 120 * time.Second, MaxRedirects: 5})\n\tresp, err := client.Get(zipURL)\n\tif err != nil {\n\t\treturn \"\", nil, fmt.Errorf(\"download zip: %w\", err)\n\t}\n\tdefer resp.Body.Close()\n\tif resp.StatusCode != http.StatusOK {\n\t\treturn \"\", nil, fmt.Errorf(\"download zip status %d\", resp.StatusCode)\n\t}\n\n\tzipData, err := io.ReadAll(resp.Body)\n\tif err != nil {\n\t\treturn \"\", nil, fmt.Errorf(\"read zip body: %w\", err)\n\t}\n\n\tzr, err := zip.NewReader(bytes.NewReader(zipData), int64(len(zipData)))\n\tif err != nil {\n\t\treturn \"\", nil, fmt.Errorf(\"open zip: %w\", err)\n\t}\n\n\t// Find .md files\n\tvar mdFiles []string","sourceCodeStart":344,"sourceCodeEnd":380,"githubUrl":"https://github.com/Tencent/WeKnora/blob/988cbb03305e055d8ebb7d46d9ac6cc0803cd074/internal/infrastructure/docparser/mineru_cloud_converter.go#L344-L380","documentation":"The HTTP GET on the MinerU Cloud result ZIP URL failed at the transport level, wrapped as 'download zip: %w' by downloadAndExtractZip (mineru_cloud_converter.go:362). This is a client.Get error: DNS failure, TLS error, connection refused/reset, redirect limit exceeded, or the SSRF-safe client's 120s timeout. The request never got a usable HTTP response.","triggerScenarios":"extractDoneResult -> downloadAndExtractZip calls the SSRF-safe client's Get on full_zip_url and the round trip fails: unreachable storage host, TLS certificate problems, more than 5 redirects, or download exceeding the 120s timeout.","commonSituations":"Expired or moved result URLs (404 handled separately — this is pre-response failure); firewall egress rules blocking the storage domain; very large ZIPs on slow links hitting the 120s timeout; redirect chains through a proxy exceeding MaxRedirects: 5.","solutions":["Read the wrapped cause: timeout vs DNS vs TLS vs redirect-limit tells you which knob to adjust.","For large files on slow links, raise the client timeout (currently 120s) in the SSRFSafeHTTPClientConfig.","Verify egress network access to the storage host from the machine running the converter.","If the URL expired, re-run the batch conversion to obtain a fresh full_zip_url."],"exampleFix":"// before\nclient := utils.NewSSRFSafeHTTPClient(utils.SSRFSafeHTTPClientConfig{Timeout: 120 * time.Second, MaxRedirects: 5})\n// after — longer timeout for large archives\nclient := utils.NewSSRFSafeHTTPClient(utils.SSRFSafeHTTPClientConfig{Timeout: 300 * time.Second, MaxRedirects: 5})","handlingStrategy":"retry","validationCode":"u, err := url.Parse(zipURL)\nif err != nil {\n    return err\n}\nconn, err := net.DialTimeout(\"tcp\", net.JoinHostPort(u.Hostname(), u.Port()), 5*time.Second)\nif err != nil {\n    return fmt.Errorf(\"storage host unreachable: %w\", err)\n}\nconn.Close()","typeGuard":null,"tryCatchPattern":"var md string\nvar images []types.ImageRef\nvar err error\nfor attempt := 0; attempt < 3; attempt++ {\n    md, images, err = pollBatchResult(ctx, batchID)\n    if err == nil || !strings.Contains(err.Error(), \"download zip:\") {\n        break\n    }\n    time.Sleep(time.Duration(attempt+1) * time.Second)\n}","preventionTips":["Confirm egress access to the MinerU result-storage host from your deployment.","Raise the client timeout for large ZIPs (default 120s) and cap redirects appropriately.","Differentiate timeout vs DNS vs TLS causes via errors.As(err, &net.Error)."],"tags":["network","http","download","timeout","go"],"backgroundTag":"http-request-failed","analyzedSha":"988cbb03305e055d8ebb7d46d9ac6cc0803cd074","analyzedAt":"2026-09-02T14:41:08.344Z","contentChangedAt":null,"schemaVersion":2},"datasetVersion":"2026-09-08T10:18:20.063Z"}