projectdiscovery/subfinder · error

error marshaling request body

Error message

error marshaling request body

What it means

Netlas source in subscraping marshals a JSON request body (fields, source_type, size) before sending its POST search query. If json.Marshal fails, the query never proceeds and an Error result with this static message is emitted. Because the payload is a plain map of strings/ints this is nearly impossible in practice, but the guard is there for safety.

Solutions

  1. Inspect the requestBody construction in pkg/subscraping/sources/netlas/netlas.go for any non-marshalable fields
  2. Rebuild against an unmodified upstream version of the netlas source file
  3. Wrap json.Marshal yourself with %w to see the underlying marshal error instead of the static message

Example fix

// before
Error: fmt.Errorf("error marshaling request body")
// after
Error: fmt.Errorf("error marshaling request body: %w", err)
Defensive patterns

Strategy: validation

Validate before calling

// Marshal is deterministic for the fixed payload; nothing to pre-validate.
if _, err := json.Marshal(requestBody); err != nil {
	// log underlying err — indicates a broken custom build
}

Try / catch

// in Go: check err from json.Marshal and surface it with %w
if err != nil {
	return fmt.Errorf("error marshaling request body: %w", err)
}

Prevention

When it happens

Trigger: json.Marshal(requestBody) returns an error while building the Netlas community-download POST body (fields:"*", source_type:"include", size=min(domainsCount.Count, communityDownloadCap)). In practice only if requestBody contains an unsupported type.

Common situations: Practically never triggered for this fixed payload; if seen it indicates a modified/custom build where the body contains channels, funcs, or cyclic structures, or a corrupted local build of the source.

Understand the failure class

Background: json.Marshal / "failed to marshal" errors in Go: why "unsupported type" happens and how to fix it — this error's family across 22 libraries.

Related errors

AI-assisted analysis of projectdiscovery/subfinder@7a0b91f0fa (2026-09-06). Data as JSON: /api/errors/b311dec85046c529. Report an issue: GitHub.

Appendix: source

Thrown at pkg/subscraping/sources/netlas/netlas.go:118

		if err != nil {
			results <- subscraping.Result{Source: s.Name(), Type: subscraping.Error, Error: err}
			s.errors++
			return
		}

		// Make a single POST request to get all domains via download method

		apiUrl := "https://app.netlas.io/api/domains/download/"
		query := fmt.Sprintf("domain:*.%s AND NOT domain:%s", domain, domain)
		requestBody := map[string]any{
			"q":           query,
			"fields":      []string{"*"},
			"source_type": "include",
			"size":        min(domainsCount.Count, communityDownloadCap),
		}
		jsonRequestBody, err := json.Marshal(requestBody)
		if err != nil {
			results <- subscraping.Result{Source: s.Name(), Type: subscraping.Error, Error: fmt.Errorf("error marshaling request body")}
			s.errors++
			return
		}

		// Pick an API key
		randomApiKey = subscraping.PickRandom(s.apiKeys, s.Name())

		s.requests++
		resp2, err := session.HTTPRequest(ctx, http.MethodPost, apiUrl, "", map[string]string{
			"accept":       "application/json",
			"X-API-Key":    randomApiKey,
			"Content-Type": "application/json"}, strings.NewReader(string(jsonRequestBody)), subscraping.BasicAuth{})
		if err != nil {
			results <- subscraping.Result{Source: s.Name(), Type: subscraping.Error, Error: err}
			s.errors++
			session.DiscardHTTPResponse(resp2)
			return
		}

View on GitHub (pinned to 7a0b91f0fa)