{"record":{"id":"48dab3054dea9a5a","repo":"projectdiscovery/katana","slug":"hybrid-response-is-nil","errorCode":null,"errorMessage":"hybrid: response is nil","messagePattern":"hybrid: response is nil","errorType":"error_code","errorClass":null,"httpStatus":null,"severity":"error","filePath":"pkg/engine/hybrid/crawl.go","lineNumber":380,"sourceCode":"\tif domErr != nil {\n\t\tgologger.Warning().Msgf(\"could not get dom for %s: %s (continuing with page HTML)\", request.URL, domErr)\n\t}\n\n\t// Use basePage with a fresh timeout for HTML retrieval so it succeeds\n\t// even if the navigation or DOM timeout was exhausted.\n\tbody, err := sessionPage.Timeout(timeout).HTML()\n\tif err != nil {\n\t\treturn nil, errkit.Wrap(err, \"hybrid: could not get html\")\n\t}\n\n\tparsed, err := urlutil.Parse(request.URL)\n\tif err != nil {\n\t\treturn nil, errkit.Wrap(err, \"hybrid: url could not be parsed\")\n\t}\n\n\tif response == nil || response.Resp == nil {\n\t\t// err is guaranteed to be nil, due to previous checks.\n\t\treturn nil, errors.New(\"hybrid: response is nil\")\n\t}\n\tresponse.Resp.Request.URL = parsed.URL\n\n\t// Create a copy of interpolated shadow DOM elements and parse them separately\n\tif domResult != nil && domResult.Root != nil {\n\t\tvar builder strings.Builder\n\t\ttraverseDOMNode(domResult.Root, &builder)\n\n\t\tresponseCopy := *response\n\t\tresponseCopy.Body = builder.String()\n\n\t\tresponseCopy.Reader, _ = goquery.NewDocumentFromReader(strings.NewReader(responseCopy.Body))\n\t\tif responseCopy.Reader != nil {\n\t\t\tnavigationRequests := c.Options.Parser.ParseResponse(&responseCopy)\n\t\t\tc.Enqueue(s.Queue, navigationRequests...)\n\t\t}\n\t}\n","sourceCodeStart":362,"sourceCodeEnd":398,"githubUrl":"https://github.com/projectdiscovery/katana/blob/e3e742739c3746f085943ce918fb4e2b8daf6fe6/pkg/engine/hybrid/crawl.go#L362-L398","documentation":"navigateRequest in the hybrid engine returns 'hybrid: response is nil' when, after issuing the HTTP request, both the response object and its underlying Resp are nil while err is nil. It is an internal invariant guard ensuring downstream code never dereferences a nil response.","triggerScenarios":"navigateRequest (crawl.go:380) after the request round-trip: response == nil || response.Resp == nil — e.g. a transport/client returning a non-error but empty result, or misconfigured response handling in custom clients.","commonSituations":"Custom HTTP clients/proxies that return (nil, nil) on certain failures; mocking layers in tests; version drift where a dependency changed its return contract.","solutions":["Inspect the HTTP client/transport in use — ensure it never returns a nil response with nil error","Update katana and dependencies to latest versions where transport behavior is fixed","Wrap custom RoundTrippers to always return either a non-nil response or a non-nil error"],"exampleFix":"// before: custom client can return (nil, nil)\nresp, _ := client.Do(req)\nreturn resp, nil\n// after\nresp, err := client.Do(req)\nif err != nil { return nil, err }\nif resp == nil { return nil, errors.New(\"empty response from client\") }\nreturn resp, nil","handlingStrategy":"type-guard","validationCode":"resp, err := client.Do(req)\nif err != nil { return nil, err }\nif resp == nil || resp.Resp == nil {\n    return nil, errors.New(\"client returned nil response\")\n}","typeGuard":"func hasValidResponse(r *types.Response) bool { return r != nil && r.Resp != nil }","tryCatchPattern":"result, err := navigateRequest(ctx, req)\nif err != nil {\n    if strings.Contains(err.Error(), \"response is nil\") { log.Warn(\"transport bug\") }\n    return nil, err\n}","preventionTips":["Wrap custom RoundTrippers to guarantee non-nil response or non-nil error","Keep katana and its HTTP dependencies updated","Avoid (nil, nil) returns in mock clients used with the hybrid engine"],"tags":["hybrid-engine","nil-response","http-client"],"backgroundTag":"nil-response","analyzedSha":"e3e742739c3746f085943ce918fb4e2b8daf6fe6","analyzedAt":"2026-09-03T14:55:13.248Z","contentChangedAt":"2026-09-03T14:55:13.248Z","schemaVersion":2},"datasetVersion":"2026-09-10T17:17:09.494Z"}