Tencent/WeKnora · error

invalid URL: %w

Error message

invalid URL: %w

What it means

ValidateURLForSSRF normalizes the target URL (prepending https:// if no scheme is present) and then parses it with url.Parse. If parsing fails — malformed percent-encodings, control characters, invalid host syntax — the error is wrapped as 'invalid URL: %w' with the underlying parse error. This is the first gate before any outbound request is allowed.

Source

Thrown at internal/utils/security.go:1193

// rawURL may be a full URL ("https://example.com/v1") or a bare host/host:port
// (for cases like ReconnectDocReader). If a scheme is missing the function
// prepends "https://" before parsing so that net/url can extract the host.
//
// Returns nil when the URL is safe, or an error describing the problem.
func ValidateURLForSSRF(rawURL string) error {
	if rawURL == "" {
		return nil // callers that require non-empty should validate separately
	}

	// Normalise: if no scheme, prepend https:// so url.Parse works correctly.
	normalized := rawURL
	if !strings.Contains(normalized, "://") {
		normalized = "https://" + normalized
	}

	parsed, err := url.Parse(normalized)
	if err != nil {
		return fmt.Errorf("invalid URL: %w", err)
	}

	hostname := parsed.Hostname()
	if hostname == "" {
		return fmt.Errorf("URL has no hostname")
	}

	// A whitelist relaxes host/IP restrictions only. It must never turn other
	// schemes (file://, gopher://, etc.) into valid outbound request targets.
	scheme := strings.ToLower(parsed.Scheme)
	if scheme != "http" && scheme != "https" {
		return fmt.Errorf("invalid scheme: %s (only http/https allowed)", scheme)
	}

	// If the host is whitelisted, skip the heavy checks.
	if IsSSRFWhitelisted(hostname) {
		return nil
	}

View on GitHub (pinned to 988cbb0330)

Solutions

1. Trim whitespace and quotes from the configured endpoint string before passing it
2. Validate with url.Parse locally and print the wrapped %w error to see the exact parse failure
3. Percent-encode reserved characters in userinfo/path portions (url.PathEscape / url.UserPassword)
4. Fix IPv6 literals to bracketed form: 'http://[::1]:9000'
5. If no scheme is desired, note the helper prepends https:// — supply a clean 'host[:port]' or full URL

Example fix

// before
client, err := newMinioClient(strings.TrimSpace(cfg.Endpoint) + "/bucket ")
// after
endpoint := strings.TrimSpace(strings.Trim(cfg.Endpoint, "\"' "))
if _, err := url.Parse(endpoint); err != nil {
    return fmt.Errorf("bad endpoint config %q: %w", endpoint, err)
}
client, err := newMinioClient(endpoint)

n := strings.TrimSpace(strings.Trim(cfg.Endpoint, "\"'"))
if !strings.Contains(n, ":/") { n = "https://" + n }
if _, err := url.Parse(n); err != nil {
    return fmt.Errorf("invalid endpoint %q: %w", cfg.Endpoint, err)
}

func isParseableURL(raw string) bool {
    n := raw
    if !strings.Contains(n, "://") { n = "https://" + n }
    _, err := url.Parse(n)
    return err == nil
}

if err := ValidateURLForSSRF(endpoint); err != nil {
    var inner error
    if errors.Unwrap(err) != nil { inner = errors.Unwrap(err) }
    log.Printf("endpoint %q rejected: %v (cause: %v)", endpoint, err, inner)
    return err
}

When it happens

Trigger: Passing a string to ValidateURLForSSRF (via any of newKS3Client, newMinioClient, NewObsFileService, CheckObsConnectivity, newOSSClient, newS3Client) that url.Parse rejects even after scheme normalization — e.g. 'http://exam ple.com' (space), 'http://[::1' (unclosed bracket), 'ht tp://x', or strings containing raw control characters.

Common situations: Object-storage endpoint config values containing stray spaces or quotes from YAML/JSON, untrimmed whitespace/newlines at the end of env vars, IPv6 literals with missing brackets, or URLs interpolated with a password containing reserved characters.

Understand the failure class

Background: "Invalid URL" / "URL cannot be empty": fix the malformed or missing URL behind request-construction failures — this error's family across 50 libraries.

Related errors

• URL has no hostname
• unsafe MinIO endpoint: %w
• unsafe TOS endpoint: %w
• %w (for private deployments on internal networks, add the ho
• invalid qqbot api_base_url: %w (for private deployments, add

AI-assisted analysis of Tencent/WeKnora@988cbb0330 (2026-09-02). Data as JSON: /api/errors/9974b76171ac373c. Report an issue: GitHub.