What causes "Response.Request is nil" in PuerkitoBio/goquery?

Assign a request to the response before parsing: res.Request = &http.Request{Method: "GET", URL: parsedURL}.

Response.Request is nil

Error message

Response.Request is nil

What it means

NewDocumentFromResponse requires res.Request (the request that generated the response) to be non-nil, because it uses res.Request.URL to resolve relative links when building the Document. If the response was constructed manually (e.g. httptest.ResponseRecorder or &http.Response{} literal) without setting Request, this error is returned.

Source

Thrown at type.go:70

	root, e := html.Parse(r)
	if e != nil {
		return nil, e
	}
	return newDocument(root, nil), nil
}

// NewDocumentFromResponse is another Document constructor that takes an http response as argument.
// It loads the specified response's document, parses it, and stores the root Document
// node, ready to be manipulated. The response's body is closed on return.
//
// Deprecated: Use goquery.NewDocumentFromReader with the response's body.
func NewDocumentFromResponse(res *http.Response) (*Document, error) {
	if res == nil {
		return nil, errors.New("Response is nil")
	}
	defer res.Body.Close()
	if res.Request == nil {
		return nil, errors.New("Response.Request is nil")
	}

	// Parse the HTML into nodes
	root, e := html.Parse(res.Body)
	if e != nil {
		return nil, e
	}

	// Create and fill the document
	return newDocument(root, res.Request.URL), nil
}

// CloneDocument creates a deep-clone of a document.
func CloneDocument(doc *Document) *Document {
	return newDocument(cloneNode(doc.rootNode), doc.Url)
}

// Private constructor, make sure all fields are correctly filled.

View on GitHub (pinned to 738783cbc3)

Solutions

Assign a request to the response before parsing: res.Request = &http.Request{Method: "GET", URL: parsedURL}.
In tests, set httptest.NewRequest(...) and recorder.Result().Request = req, or use http.Client against httptest.Server so Request is populated.
Use goquery.NewDocumentFromReader(res.Body) if you do not need URL-based relative-link resolution.

Example fix

// before
rec := httptest.NewRecorder()
rec.WriteString("<html></html>")
doc, err := goquery.NewDocumentFromResponse(rec.Result()) // Request is nil

// after
req := httptest.NewRequest("GET", "http://example.com/", nil)
rec := httptest.NewRecorder()
rec.WriteString("<html></html>")
res := rec.Result()
res.Request = req
doc, err := goquery.NewDocumentFromResponse(res)

Defensive patterns

Strategy: validation

Validate before calling

if res == nil || res.Request == nil || res.Request.URL == nil {
	return errors.New("response lacks Request/URL context")
}
doc, err := goquery.NewDocumentFromResponse(res)

Type guard

func hasRequestContext(res *http.Response) bool {
	return res != nil && res.Request != nil && res.Request.URL != nil
}

Prevention

In tests using httptest.ResponseRecorder, always attach res.Request = httptest.NewRequest(...).
Never hand-build http.Response literals without setting Request.URL.
Mock at the http.RoundTripper/httptest.Server level so responses carry their originating request.

When it happens

Trigger: Passing an *http.Response that was hand-constructed or produced by httptest.NewRecorder().Result() without assigning res.Request; responses from clients that do not populate Request.

Common situations: Unit tests building fake responses with httptest where only Body/StatusCode were set; deserialized responses from caches or mocks; constructing a response from a saved payload and forgetting Request.URL.

Understand the failure class

Background: "missing required argument" and "the following required arguments were not provided": what required-argument errors mean and how to fix them — this error's family across 20 libraries.

Related errors

AI-assisted analysis of PuerkitoBio/goquery@738783cbc3 (2026-09-06). Data as JSON: /api/errors/8a4ffcdf7995db89. Report an issue: GitHub.