{"record":{"id":"c65b068dd8b2a76b","repo":"Tencent/WeKnora","slug":"zip-url-blocked-by-ssrf-check-v","errorCode":null,"errorMessage":"zip URL blocked by SSRF check: %v","messagePattern":"zip URL blocked by SSRF check: (.+?)","errorType":"validation","errorClass":null,"httpStatus":null,"severity":"error","filePath":"internal/infrastructure/docparser/mineru_cloud_converter.go","lineNumber":357,"sourceCode":"\t\treturn \"\", nil, fmt.Errorf(\"MinerU Cloud state=done but no markdown/content or full_zip_url\")\n\t}\n\n\tmd, imageRefs, err := downloadAndExtractZip(item.FullZipURL)\n\tif err != nil {\n\t\treturn \"\", nil, fmt.Errorf(\"extract zip: %w\", err)\n\t}\n\n\tlogger.Infof(context.Background(), \"[MinerUCloud] parsed (zip), markdown=%d chars, images=%d\", len(md), len(imageRefs))\n\treturn md, imageRefs, nil\n}\n\n// --- ZIP handling ---\n\nvar imgRefPattern = regexp.MustCompile(`!\\[[^\\]]*\\]\\(([^)]+)\\)`)\n\nfunc downloadAndExtractZip(zipURL string) (string, []types.ImageRef, error) {\n\tif err := utils.ValidateURLForSSRF(zipURL); err != nil {\n\t\treturn \"\", nil, fmt.Errorf(\"zip URL blocked by SSRF check: %v\", err)\n\t}\n\tclient := utils.NewSSRFSafeHTTPClient(utils.SSRFSafeHTTPClientConfig{Timeout: 120 * time.Second, MaxRedirects: 5})\n\tresp, err := client.Get(zipURL)\n\tif err != nil {\n\t\treturn \"\", nil, fmt.Errorf(\"download zip: %w\", err)\n\t}\n\tdefer resp.Body.Close()\n\tif resp.StatusCode != http.StatusOK {\n\t\treturn \"\", nil, fmt.Errorf(\"download zip status %d\", resp.StatusCode)\n\t}\n\n\tzipData, err := io.ReadAll(resp.Body)\n\tif err != nil {\n\t\treturn \"\", nil, fmt.Errorf(\"read zip body: %w\", err)\n\t}\n\n\tzr, err := zip.NewReader(bytes.NewReader(zipData), int64(len(zipData)))\n\tif err != nil {","sourceCodeStart":339,"sourceCodeEnd":375,"githubUrl":"https://github.com/Tencent/WeKnora/blob/988cbb03305e055d8ebb7d46d9ac6cc0803cd074/internal/infrastructure/docparser/mineru_cloud_converter.go#L339-L375","documentation":"The full_zip_url returned by MinerU Cloud was rejected by utils.ValidateURLForSSRF before download in downloadAndExtractZip (mineru_cloud_converter.go:357). The library blocks URLs that resolve to private/loopback/link-local addresses or disallowed schemes to prevent server-side request forgery. If the URL is a legitimate public storage link, this indicates an over-restrictive SSRF policy or a DNS/environment quirk (e.g. internal DNS resolving the storage host to a private IP).","triggerScenarios":"extractDoneResult -> downloadAndExtractZip receives a zipURL that fails SSRF validation: non-http(s) scheme, localhost/loopback host, private or link-local IP resolution, metadata endpoints (169.254.169.254), or disallowed ports.","commonSituations":"Self-hosted/proxied MinerU deployments whose result URLs point at internal storage; test environments using local mock URLs; containers where the storage host resolves to a cluster-internal IP; misconfigured public base URL falling back to an internal address.","solutions":["Inspect the wrapped %v cause to see which SSRF rule fired (scheme, private IP, port).","If the storage host is legitimately internal, add it to the SSRF allowlist or configure the allowed CIDR ranges in the SSRF client config.","Ensure MinerU Cloud returns publicly resolvable full_zip_url values (check the deployment's public endpoint configuration).","Never bypass SSRF checks for untrusted URLs; only allowlist verified internal storage hosts."],"exampleFix":"// before — blanket bypass (unsafe)\n// md, imageRefs, err := download zip without check\n// after — allowlist the trusted internal storage CIDR\nutils.NewSSRFSafeHTTPClient(utils.SSRFSafeHTTPClientConfig{\n    Timeout:        120 * time.Second,\n    MaxRedirects:   5,\n    AllowedCIDRs:   []string{\"10.0.0.0/8\"}, // only if storage is trusted-internal\n})","handlingStrategy":"validation","validationCode":"u, err := url.Parse(zipURL)\nif err != nil {\n    return err\n}\nif u.Scheme != \"https\" && u.Scheme != \"http\" {\n    return fmt.Errorf(\"unsupported scheme: %s\", u.Scheme)\n}\nips, err := net.LookupIP(u.Hostname())\nif err != nil {\n    return err\n}\nfor _, ip := range ips {\n    if ip.IsLoopback() || ip.IsPrivate() || ip.IsLinkLocalUnicast() {\n        return fmt.Errorf(\"zip URL resolves to non-public address: %v\", ip)\n    }\n}","typeGuard":"func isPublicHTTPURL(raw string) bool {\n    u, err := url.Parse(raw)\n    if err != nil || (u.Scheme != \"http\" && u.Scheme != \"https\") {\n        return false\n    }\n    ips, err := net.LookupIP(u.Hostname())\n    if err != nil {\n        return false\n    }\n    for _, ip := range ips {\n        if ip.IsLoopback() || ip.IsPrivate() || ip.IsLinkLocalUnicast() {\n            return false\n        }\n    }\n    return true\n}","tryCatchPattern":"md, images, err := pollBatchResult(ctx, batchID)\nif err != nil {\n    if strings.Contains(err.Error(), \"zip URL blocked by SSRF check\") {\n        // expected for internal URLs: use an allowlisted fetcher or fail fast\n        return fmt.Errorf(\"result URL not fetchable from this network: %w\", err)\n    }\n    return err\n}","preventionTips":["Pre-validate zip URLs for scheme and public resolvability before conversion flows that require zip download.","If result storage is internal, configure the SSRF allowlist explicitly — never disable the check.","Ensure the MinerU deployment exposes publicly resolvable full_zip_url values."],"tags":["ssrf","security","url-validation","go","network"],"backgroundTag":"ssrf-url-blocked","analyzedSha":"988cbb03305e055d8ebb7d46d9ac6cc0803cd074","analyzedAt":"2026-09-02T14:41:08.344Z","contentChangedAt":null,"schemaVersion":2},"datasetVersion":"2026-09-08T10:18:20.063Z"}