{"record":{"id":"9974b76171ac373c","repo":"Tencent/WeKnora","slug":"invalid-url-w-9974b7","errorCode":null,"errorMessage":"invalid URL: %w","messagePattern":"invalid URL: %w","errorType":"validation","errorClass":null,"httpStatus":null,"severity":"error","filePath":"internal/utils/security.go","lineNumber":1193,"sourceCode":"// rawURL may be a full URL (\"https://example.com/v1\") or a bare host/host:port\n// (for cases like ReconnectDocReader). If a scheme is missing the function\n// prepends \"https://\" before parsing so that net/url can extract the host.\n//\n// Returns nil when the URL is safe, or an error describing the problem.\nfunc ValidateURLForSSRF(rawURL string) error {\n\tif rawURL == \"\" {\n\t\treturn nil // callers that require non-empty should validate separately\n\t}\n\n\t// Normalise: if no scheme, prepend https:// so url.Parse works correctly.\n\tnormalized := rawURL\n\tif !strings.Contains(normalized, \"://\") {\n\t\tnormalized = \"https://\" + normalized\n\t}\n\n\tparsed, err := url.Parse(normalized)\n\tif err != nil {\n\t\treturn fmt.Errorf(\"invalid URL: %w\", err)\n\t}\n\n\thostname := parsed.Hostname()\n\tif hostname == \"\" {\n\t\treturn fmt.Errorf(\"URL has no hostname\")\n\t}\n\n\t// A whitelist relaxes host/IP restrictions only. It must never turn other\n\t// schemes (file://, gopher://, etc.) into valid outbound request targets.\n\tscheme := strings.ToLower(parsed.Scheme)\n\tif scheme != \"http\" && scheme != \"https\" {\n\t\treturn fmt.Errorf(\"invalid scheme: %s (only http/https allowed)\", scheme)\n\t}\n\n\t// If the host is whitelisted, skip the heavy checks.\n\tif IsSSRFWhitelisted(hostname) {\n\t\treturn nil\n\t}","sourceCodeStart":1175,"sourceCodeEnd":1211,"githubUrl":"https://github.com/Tencent/WeKnora/blob/988cbb03305e055d8ebb7d46d9ac6cc0803cd074/internal/utils/security.go#L1175-L1211","documentation":"ValidateURLForSSRF normalizes the target URL (prepending https:// if no scheme is present) and then parses it with url.Parse. If parsing fails — malformed percent-encodings, control characters, invalid host syntax — the error is wrapped as 'invalid URL: %w' with the underlying parse error. This is the first gate before any outbound request is allowed.","triggerScenarios":"Passing a string to ValidateURLForSSRF (via any of newKS3Client, newMinioClient, NewObsFileService, CheckObsConnectivity, newOSSClient, newS3Client) that url.Parse rejects even after scheme normalization — e.g. 'http://exam ple.com' (space), 'http://[::1' (unclosed bracket), 'ht tp://x', or strings containing raw control characters.","commonSituations":"Object-storage endpoint config values containing stray spaces or quotes from YAML/JSON, untrimmed whitespace/newlines at the end of env vars, IPv6 literals with missing brackets, or URLs interpolated with a password containing reserved characters.","solutions":["Trim whitespace and quotes from the configured endpoint string before passing it","Validate with url.Parse locally and print the wrapped %w error to see the exact parse failure","Percent-encode reserved characters in userinfo/path portions (url.PathEscape / url.UserPassword)","Fix IPv6 literals to bracketed form: 'http://[::1]:9000'","If no scheme is desired, note the helper prepends https:// — supply a clean 'host[:port]' or full URL"],"exampleFix":"// before\nclient, err := newMinioClient(strings.TrimSpace(cfg.Endpoint) + \"/bucket \")\n// after\nendpoint := strings.TrimSpace(strings.Trim(cfg.Endpoint, \"\\\"' \"))\nif _, err := url.Parse(endpoint); err != nil {\n    return fmt.Errorf(\"bad endpoint config %q: %w\", endpoint, err)\n}\nclient, err := newMinioClient(endpoint)","handlingStrategy":"validation","validationCode":"n := strings.TrimSpace(strings.Trim(cfg.Endpoint, \"\\\"'\"))\nif !strings.Contains(n, \":/\") { n = \"https://\" + n }\nif _, err := url.Parse(n); err != nil {\n    return fmt.Errorf(\"invalid endpoint %q: %w\", cfg.Endpoint, err)\n}","typeGuard":"func isParseableURL(raw string) bool {\n    n := raw\n    if !strings.Contains(n, \"://\") { n = \"https://\" + n }\n    _, err := url.Parse(n)\n    return err == nil\n}","tryCatchPattern":"if err := ValidateURLForSSRF(endpoint); err != nil {\n    var inner error\n    if errors.Unwrap(err) != nil { inner = errors.Unwrap(err) }\n    log.Printf(\"endpoint %q rejected: %v (cause: %v)\", endpoint, err, inner)\n    return err\n}","preventionTips":["Trim whitespace/quotes/newlines from endpoint config values","Percent-encode credentials and special characters in URLs","Use bracketed IPv6 literals: http://[::1]:9000","Pre-validate endpoint strings in config loading, before client construction"],"tags":["ssrf","url-parsing","configuration","object-storage"],"backgroundTag":"invalid-url","analyzedSha":"988cbb03305e055d8ebb7d46d9ac6cc0803cd074","analyzedAt":"2026-09-02T14:41:08.344Z","contentChangedAt":null,"schemaVersion":2},"datasetVersion":"2026-09-08T10:18:20.063Z"}