projectdiscovery/subfinder · error

unexpected status code

Error message

unexpected status code %d received from %s

What it means

httpRequestWrapper treats HTTP responses with unexpected (non-success) status codes as failures and returns this error along with the response. The wrapper logs the failed response body at debug level, so the error signals a bad upstream status during source scraping.

Solutions

1. Check the status code in the error: 401/403 → fix API keys, 429 → slow down or add rate limiting, 5xx → retry later
2. Configure valid API keys for the failing source in the provider config
3. Apply rate limiting (-rls) for the affected source and retry
4. Temporarily disable the failing source if its data is not critical

resp, err := http.Head(sourceURL)
if err == nil && resp.StatusCode >= 200 && resp.StatusCode < 300 {
    // safe to query
}

resp, err := agent.HTTPRequest(ctx, req)
if err != nil {
    var status int
    if _, serr := fmt.Sscanf(err.Error(), "unexpected status code %d", &status); serr == nil {
        switch {
        case status == 429:
            // backoff and retry
        case status == 401 || status == 403:
            // fix API key
        default:
            // retry later or skip source
        }
    }
}

When it happens

Trigger: Any source HTTP request (via HTTPRequest or TestHTTPRequestWrapper) that returns a status code outside the accepted range — e.g. 403 for missing/invalid API keys, 429 rate limiting, 5xx server errors.

Common situations: Expired or missing API keys for passive sources; exceeding a provider's rate limits; provider temporarily down; IP blocked by the data source.

Understand the failure class

Background: "API error: {status}" and "HTTP 401/403/404/429/5xx" errors: non-2xx HTTP responses explained — this error's family across 27 libraries.

Related errors

• %s
• request failed with status
• request failed with status
• error reading response body
• unexpected status code

AI-assisted analysis of projectdiscovery/subfinder@7a0b91f0fa (2026-09-06). Data as JSON: /api/errors/3c6a5627206c2a0e. Report an issue: GitHub.

Appendix: source

Thrown at pkg/subscraping/agent.go:177

func httpRequestWrapper(client *http.Client, request *http.Request, maxResponseBodySize int64) (*http.Response, error) {
	response, err := client.Do(request)
	if err != nil {
		return nil, err
	}

	// Optionally cap the untrusted response body centrally so every source
	// (and the debug-logging path below) inherits the same bound when enabled.
	LimitResponseBody(response, maxResponseBodySize)

	if response.StatusCode != http.StatusOK {
		requestURL, _ := url.QueryUnescape(request.URL.String())

		gologger.Debug().MsgFunc(func() string {
			buffer := new(bytes.Buffer)
			_, _ = buffer.ReadFrom(response.Body)
			return fmt.Sprintf("Response for failed request against %s:\n%s", requestURL, buffer.String())
		})
		return response, fmt.Errorf("unexpected status code %d received from %s", response.StatusCode, requestURL)
	}
	return response, nil
}

View on GitHub (pinned to 7a0b91f0fa)