{"record":{"id":"91d91768c8ee3366","repo":"Tencent/WeKnora","slug":"unsupported-url-scheme-s","errorCode":null,"errorMessage":"unsupported URL scheme: %s","messagePattern":"unsupported URL scheme: (.+?)","errorType":"http","errorClass":null,"httpStatus":null,"severity":"error","filePath":"internal/utils/httputil.go","lineNumber":20,"sourceCode":"\nimport (\n\t\"fmt\"\n\t\"io\"\n\t\"net/http\"\n\t\"strings\"\n\t\"time\"\n)\n\nvar defaultHTTPClient = NewSSRFSafeHTTPClient(SSRFSafeHTTPClientConfig{\n\tTimeout:      60 * time.Second,\n\tMaxRedirects: 10,\n})\n\n// DownloadBytes fetches the content at the given HTTP(S) URL and returns the\n// raw bytes. It reuses a package-level http.Client with a 60-second timeout.\nfunc DownloadBytes(url string) ([]byte, error) {\n\tif !strings.HasPrefix(url, \"http://\") && !strings.HasPrefix(url, \"https://\") {\n\t\treturn nil, fmt.Errorf(\"unsupported URL scheme: %s\", url)\n\t}\n\tif err := ValidateURLForSSRF(url); err != nil {\n\t\treturn nil, fmt.Errorf(\"URL rejected by SSRF policy: %w\", err)\n\t}\n\tresp, err := defaultHTTPClient.Get(url)\n\tif err != nil {\n\t\treturn nil, fmt.Errorf(\"HTTP GET: %w\", err)\n\t}\n\tdefer resp.Body.Close()\n\tif resp.StatusCode != http.StatusOK {\n\t\treturn nil, fmt.Errorf(\"HTTP %d for %s\", resp.StatusCode, url)\n\t}\n\tdata, err := io.ReadAll(resp.Body)\n\tif err != nil {\n\t\treturn nil, fmt.Errorf(\"read body: %w\", err)\n\t}\n\treturn data, nil\n}","sourceCodeStart":2,"sourceCodeEnd":38,"githubUrl":"https://github.com/Tencent/WeKnora/blob/988cbb03305e055d8ebb7d46d9ac6cc0803cd074/internal/utils/httputil.go#L2-L38","documentation":"DownloadBytes only accepts absolute http:// or https:// URLs; anything else (ftp:, file:, ws:, or scheme-less strings) is rejected before any network activity. This guard ensures the function only performs HTTP GETs.","triggerScenarios":"Calling DownloadBytes with a URL lacking an http(s) prefix: file:// paths, data: URIs, relative paths like \"/api/x\", hostnames without a scheme (\"example.com/file\"), or other schemes (ftp, s3).","commonSituations":"Passing a local file path expecting the helper to read disk, config storing scheme-less URLs, URLs taken from user input without normalization, or S3/blob-store URIs handed to a generic downloader.","solutions":["Prefix the URL with https:// (or http://) before calling DownloadBytes","For local files, read with os.ReadFile instead of DownloadBytes","Normalize/complete the URL when it comes from config or user input","If other schemes are needed, resolve them separately and hand DownloadBytes only the https endpoint"],"exampleFix":"// before\nDownloadBytes(\"example.com/data.json\")\n// after\nDownloadBytes(\"https://example.com/data.json\")","handlingStrategy":"validation","validationCode":"u, err := url.Parse(raw)\nif err != nil || (u.Scheme != \"http\" && u.Scheme != \"https\") {\n    return fmt.Errorf(\"must be an http(s) URL\")\n}","typeGuard":"func isHTTPURL(raw string) bool {\n    u, err := url.Parse(raw)\n    return err == nil && (u.Scheme == \"http\" || u.Scheme == \"https\") && u.Host != \"\"\n}","tryCatchPattern":"data, err := DownloadBytes(url)\nif err != nil {\n    if strings.Contains(err.Error(), \"unsupported URL scheme\") {\n        // normalize URL or fall back to a file/scheme-specific reader\n    }\n}","preventionTips":["Always store fully-qualified http(s) URLs in config","Normalize scheme-less user input by defaulting to https://","Use os.ReadFile for local files instead of a URL downloader"],"tags":["http","url","validation","go"],"backgroundTag":"unsupported-url-scheme","analyzedSha":"988cbb03305e055d8ebb7d46d9ac6cc0803cd074","analyzedAt":"2026-09-02T14:41:08.344Z","contentChangedAt":null,"schemaVersion":2},"datasetVersion":"2026-09-08T10:18:20.063Z"}