shadow1ng/fscan · error

%s: %w (webscan_http_request_error)

Error message

%s: %w (webscan_http_request_error)

What it means

clustersend (invoked from clusterpoc for brute-force/multi-set POCs) builds a new HTTP request with http.NewRequestWithContext using scheme://host+path composed from the parsed request. When that construction fails, the error is wrapped with the localized 'webscan_http_request_error' message and the underlying cause. It signals the probe URL or HTTP method is not valid per net/http before any traffic is sent.

Source

Thrown at webscan/lib/poc_executor.go:788

		rule.Path = strings.ReplaceAll(strings.TrimSpace(rule.Path), varPattern, strValue)
		rule.Body = strings.ReplaceAll(strings.TrimSpace(rule.Body), varPattern, strValue)
	}

	// 构建完整请求路径
	if oReq.URL.Path != "" && oReq.URL.Path != "/" {
		req.URL.Path = fmt.Sprint(oReq.URL.Path, rule.Path)
	} else {
		req.URL.Path = rule.Path
	}

	// URL编码处理
	req.URL.Path = strings.ReplaceAll(req.URL.Path, " ", "%20")

	// 创建新的HTTP请求（传递原始请求的Context以支持超时控制）
	reqURL := fmt.Sprintf("%s://%s%s", req.URL.Scheme, req.URL.Host, req.URL.Path)
	newRequest, err := http.NewRequestWithContext(oReq.Context(), rule.Method, reqURL, strings.NewReader(rule.Body))
	if err != nil {
		return false, fmt.Errorf("%s: %w", i18n.GetText("webscan_http_request_error"), err)
	}
	defer func() { newRequest = nil }()

	// 设置请求头
	newRequest.Header = oReq.Header.Clone()
	for key, value := range rule.Headers {
		newRequest.Header.Set(key, value)
	}

	// 发送请求
	resp, err := DoRequest(newRequest, rule.FollowRedirects, session)
	if err != nil {
		if isTransportError(err) {
			return false, nil
		}
		return false, fmt.Errorf("%s: %w", i18n.GetText("webscan_request_send_error"), err)
	}

View on GitHub (pinned to 95cc12e753)

Solutions

1. Check the wrapped cause: "net/http: invalid method" points at rule.Method; "parse ... invalid URI" or "empty url" points at the composed reqURL.
2. Sanitize cluster-set variable values before substitution (strip control characters, escape path segments).
3. Verify the original request's URL.Scheme and URL.Host are non-empty and valid before calling clustersend.
4. Fix the POC rule definition — typo'd method or path with illegal characters is the usual root cause.

Example fix

// before
newRequest, err := http.NewRequestWithContext(oReq.Context(), rule.Method, reqURL, strings.NewReader(rule.Body))
// after
if !isValidMethod(rule.Method) {
    return false, fmt.Errorf("invalid rule method %q", rule.Method)
}
reqURL = sanitizeURL(reqURL)
newRequest, err := http.NewRequestWithContext(oReq.Context(), rule.Method, reqURL, strings.NewReader(rule.Body))

func validClusterRule(rule Rules, req *Request) error {
    if !isValidToken(rule.Method) {
        return fmt.Errorf("invalid method %q", rule.Method)
    }
    reqURL := fmt.Sprintf("%s://%s%s", req.URL.Scheme, req.URL.Host,
        strings.ReplaceAll(req.URL.Path, " ", "%20"))
    _, err := url.Parse(reqURL)
    return err
}

func canBuildRequest(ctx context.Context, method, rawURL string, body string) bool {
    _, err := http.NewRequestWithContext(ctx, method, rawURL, strings.NewReader(body))
    return err == nil
}

ok, err := clusterpoc(oReq, p, vars, req, env, ctx, cache)
if err != nil {
    if strings.Contains(err.Error(), i18n.GetText("webscan_http_request_error")) {
        log.Printf("cluster rule for %s built an invalid request: %v", p.Name, err)
        return
    }
    return err
}

When it happens

Trigger: In clusterpoc mode, a rule's method string or the composed reqURL (req.URL.Scheme + req.URL.Host + req.URL.Path after space-to-%20 replacement) is rejected by http.NewRequestWithContext — e.g. method with spaces/invalid chars, URL containing control characters or an unparsable host.

Common situations: Batch POCs with per-set variables whose substituted values contain raw newlines, quotes, or braces that corrupt the URL; a target whose Host header/URL was crafted with characters url.Parse tolerates but NewRequest rejects; miswritten cluster rule methods ("Post" with trailing whitespace is fine, "GE T" is not).

Understand the failure class

Background: "Invalid URL" errors: why new URL(), URI.parse, and reqwest::Url reject your string — missing scheme, whitespace, and bad path format — this error's family across 39 libraries.

Related errors

• %s: %w (webscan_request_create_error)
• webscan_request_body_read_failed
• network_rate_limited
• %s: %w (webscan_request_send_error)
• webscan_poc_convert_failed

AI-assisted analysis of shadow1ng/fscan@95cc12e753 (2026-09-06). Data as JSON: /api/errors/c94c7c6778fb60e1. Report an issue: GitHub.