{"record":{"id":"c94c7c6778fb60e1","repo":"shadow1ng/fscan","slug":"s-w-webscan-http-request-error","errorCode":null,"errorMessage":"%s: %w (webscan_http_request_error)","messagePattern":"(.+?): %w \\(webscan_http_request_error\\)","errorType":"exception","errorClass":null,"httpStatus":null,"severity":"error","filePath":"webscan/lib/poc_executor.go","lineNumber":788,"sourceCode":"\t\trule.Path = strings.ReplaceAll(strings.TrimSpace(rule.Path), varPattern, strValue)\n\t\trule.Body = strings.ReplaceAll(strings.TrimSpace(rule.Body), varPattern, strValue)\n\t}\n\n\t// 构建完整请求路径\n\tif oReq.URL.Path != \"\" && oReq.URL.Path != \"/\" {\n\t\treq.URL.Path = fmt.Sprint(oReq.URL.Path, rule.Path)\n\t} else {\n\t\treq.URL.Path = rule.Path\n\t}\n\n\t// URL编码处理\n\treq.URL.Path = strings.ReplaceAll(req.URL.Path, \" \", \"%20\")\n\n\t// 创建新的HTTP请求（传递原始请求的Context以支持超时控制）\n\treqURL := fmt.Sprintf(\"%s://%s%s\", req.URL.Scheme, req.URL.Host, req.URL.Path)\n\tnewRequest, err := http.NewRequestWithContext(oReq.Context(), rule.Method, reqURL, strings.NewReader(rule.Body))\n\tif err != nil {\n\t\treturn false, fmt.Errorf(\"%s: %w\", i18n.GetText(\"webscan_http_request_error\"), err)\n\t}\n\tdefer func() { newRequest = nil }()\n\n\t// 设置请求头\n\tnewRequest.Header = oReq.Header.Clone()\n\tfor key, value := range rule.Headers {\n\t\tnewRequest.Header.Set(key, value)\n\t}\n\n\t// 发送请求\n\tresp, err := DoRequest(newRequest, rule.FollowRedirects, session)\n\tif err != nil {\n\t\tif isTransportError(err) {\n\t\t\treturn false, nil\n\t\t}\n\t\treturn false, fmt.Errorf(\"%s: %w\", i18n.GetText(\"webscan_request_send_error\"), err)\n\t}\n","sourceCodeStart":770,"sourceCodeEnd":806,"githubUrl":"https://github.com/shadow1ng/fscan/blob/95cc12e753bf43de7004e5aef42a9ffba3934303/webscan/lib/poc_executor.go#L770-L806","documentation":"clustersend (invoked from clusterpoc for brute-force/multi-set POCs) builds a new HTTP request with http.NewRequestWithContext using scheme://host+path composed from the parsed request. When that construction fails, the error is wrapped with the localized 'webscan_http_request_error' message and the underlying cause. It signals the probe URL or HTTP method is not valid per net/http before any traffic is sent.","triggerScenarios":"In clusterpoc mode, a rule's method string or the composed reqURL (req.URL.Scheme + req.URL.Host + req.URL.Path after space-to-%20 replacement) is rejected by http.NewRequestWithContext — e.g. method with spaces/invalid chars, URL containing control characters or an unparsable host.","commonSituations":"Batch POCs with per-set variables whose substituted values contain raw newlines, quotes, or braces that corrupt the URL; a target whose Host header/URL was crafted with characters url.Parse tolerates but NewRequest rejects; miswritten cluster rule methods (\"Post\" with trailing whitespace is fine, \"GE T\" is not).","solutions":["Check the wrapped cause: \"net/http: invalid method\" points at rule.Method; \"parse ... invalid URI\" or \"empty url\" points at the composed reqURL.","Sanitize cluster-set variable values before substitution (strip control characters, escape path segments).","Verify the original request's URL.Scheme and URL.Host are non-empty and valid before calling clustersend.","Fix the POC rule definition — typo'd method or path with illegal characters is the usual root cause."],"exampleFix":"// before\nnewRequest, err := http.NewRequestWithContext(oReq.Context(), rule.Method, reqURL, strings.NewReader(rule.Body))\n// after\nif !isValidMethod(rule.Method) {\n    return false, fmt.Errorf(\"invalid rule method %q\", rule.Method)\n}\nreqURL = sanitizeURL(reqURL)\nnewRequest, err := http.NewRequestWithContext(oReq.Context(), rule.Method, reqURL, strings.NewReader(rule.Body))","handlingStrategy":"validation","validationCode":"func validClusterRule(rule Rules, req *Request) error {\n    if !isValidToken(rule.Method) {\n        return fmt.Errorf(\"invalid method %q\", rule.Method)\n    }\n    reqURL := fmt.Sprintf(\"%s://%s%s\", req.URL.Scheme, req.URL.Host,\n        strings.ReplaceAll(req.URL.Path, \" \", \"%20\"))\n    _, err := url.Parse(reqURL)\n    return err\n}","typeGuard":"func canBuildRequest(ctx context.Context, method, rawURL string, body string) bool {\n    _, err := http.NewRequestWithContext(ctx, method, rawURL, strings.NewReader(body))\n    return err == nil\n}","tryCatchPattern":"ok, err := clusterpoc(oReq, p, vars, req, env, ctx, cache)\nif err != nil {\n    if strings.Contains(err.Error(), i18n.GetText(\"webscan_http_request_error\")) {\n        log.Printf(\"cluster rule for %s built an invalid request: %v\", p.Name, err)\n        return\n    }\n    return err\n}","preventionTips":["Sanitize all cluster-set variable values (strip control chars, escape paths) before substitution","Use url.PathEscape on dynamic path segments","Validate the composed scheme://host+path URL with url.Parse before NewRequestWithContext","Keep brute-force payloads free of newlines/quotes"],"tags":["go","webscan","http","request-construction"],"backgroundTag":"invalid-url-format","analyzedSha":"95cc12e753bf43de7004e5aef42a9ffba3934303","analyzedAt":"2026-09-06T17:07:30.094Z","contentChangedAt":"2026-09-06T17:07:30.094Z","schemaVersion":2},"datasetVersion":"2026-09-14T00:17:10.932Z"}