{"record":{"id":"ac47f183087e16a6","repo":"projectdiscovery/katana","slug":"could-not-create-new-request","errorCode":null,"errorMessage":"could not create new request","messagePattern":"could not create new request","errorType":"exception","errorClass":null,"httpStatus":null,"severity":"warning","filePath":"pkg/engine/headless/browser/browser.go","lineNumber":625,"sourceCode":"\tif err != nil {\n\t\treturn nil, err\n\t}\n\n\tif !r.Base64Encoded {\n\t\treturn []byte(r.Body), nil\n\t}\n\n\tbs, err := base64.StdEncoding.DecodeString(r.Body)\n\tif err != nil {\n\t\treturn nil, err\n\t}\n\treturn bs, nil\n}\n\nfunc netHTTPRequestFromProto(e *proto.NetworkRequest) (*http.Request, error) {\n\treq, err := http.NewRequest(e.Method, e.URL, nil)\n\tif err != nil {\n\t\treturn nil, errors.Wrap(err, \"could not create new request\")\n\t}\n\tfor k, v := range e.Headers {\n\t\treq.Header.Set(k, v.Str())\n\t}\n\tif e.PostData != \"\" {\n\t\treq.Body = io.NopCloser(strings.NewReader(e.PostData))\n\t\treq.ContentLength = int64(len(e.PostData))\n\t}\n\treturn req, nil\n}\n\nfunc netHTTPResponseFromProto(e *proto.FetchRequestPaused, body []byte) *http.Response {\n\thttpresp := &http.Response{\n\t\tProto:         \"HTTP/1.1\",\n\t\tProtoMajor:    1,\n\t\tProtoMinor:    1,\n\t\tHeader:        make(http.Header),\n\t\tStatusCode:    *e.ResponseStatusCode,","sourceCodeStart":607,"sourceCodeEnd":643,"githubUrl":"https://github.com/projectdiscovery/katana/blob/e3e742739c3746f085943ce918fb4e2b8daf6fe6/pkg/engine/headless/browser/browser.go#L607-L643","documentation":"netHTTPRequestFromProto converts a CDP proto.NetworkRequest (captured by the Fetch domain interceptor) into a Go *http.Request, and wraps http.NewRequest failures with this message. http.NewRequest only fails on an unparseable URL or an invalid HTTP method, so this error indicates a malformed request URL or method arriving from the browser event stream. The caller (the FetchRequestPaused handler) silently drops the request when this happens.","triggerScenarios":"e.URL being empty, relative, or otherwise unparseable (e.g. 'blob:', 'data:', malformed absolute URLs, or missing scheme) or e.Method being an invalid token, when a paused response event is converted for the RequestCallback pipeline.","commonSituations":"Crawling pages that issue requests to non-HTTP schemes (blob:, data:, chrome-extension:) that CDP reports but net/url cannot parse as absolute HTTP URLs; redirect chains or service workers producing URLs without a host; corrupted/empty NetworkRequest fields from unusual resources like WebSocket or manifest fetches.","solutions":["Filter non-HTTP(S) URL schemes (blob:, data:, chrome-extension:, about:) from e.URL before calling netHTTPRequestFromProto.","Validate that e.URL parses with url.Parse and has http/https scheme, and that e.Method is a valid HTTP token; skip conversion otherwise.","If URLs legitimately lack a host due to redirects, resolve them against the page's base URL before conversion.","This error is swallowed by the handler (returns silently) — if you need those requests, add logging at the call site (browser.go:550)."],"exampleFix":"// before\nfunc netHTTPRequestFromProto(e *proto.NetworkRequest) (*http.Request, error) {\n    req, err := http.NewRequest(e.Method, e.URL, nil)\n    ...\n}\n\n// after: skip non-HTTP schemes upstream\nu, perr := url.Parse(e.Request.URL)\nif perr != nil || (u.Scheme != \"http\" && u.Scheme != \"https\") {\n    return // not convertible; skip callback emission\n}\nhttpreq, err := netHTTPRequestFromProto(e.Request)","handlingStrategy":"validation","validationCode":"func isConvertibleNetworkRequest(e *proto.NetworkRequest) bool {\n    u, err := url.Parse(e.URL)\n    return err == nil && u.IsAbs() && (u.Scheme == \"http\" || u.Scheme == \"https\") && e.Method != \"\"\n}","typeGuard":"// Go-style guard used before conversion\nif !isConvertibleNetworkRequest(e.Request) {\n    return // skip non-HTTP/unparseable intercepted requests\n}","tryCatchPattern":"httpreq, err := netHTTPRequestFromProto(e.Request)\nif err != nil {\n    slog.Debug(\"skipping non-convertible intercepted request\", \"url\", e.Request.URL, \"error\", err)\n    return\n}","preventionTips":["Always scheme-check intercepted URLs (allow only http/https) before conversion.","Log dropped requests so silently skipped callbacks are observable.","Resolve relative or scheme-less URLs against the page's base URL when possible.","Sanitize/validate e.Method against a known HTTP method set."],"tags":["http","url-parsing","cdp","request-interception"],"backgroundTag":"invalid-url-parse","analyzedSha":"e3e742739c3746f085943ce918fb4e2b8daf6fe6","analyzedAt":"2026-09-03T14:55:13.248Z","contentChangedAt":"2026-09-03T14:55:13.248Z","schemaVersion":2},"datasetVersion":"2026-09-10T17:17:09.494Z"}