What causes "unexpected status code: %d" in projectdiscovery/subfinder?

Check resp.StatusCode and apply exponential backoff, especially for 429.

unexpected status code

Error message

unexpected status code: %d

What it means

The threatcrowd source raises this when the HTTP response status is not 200 OK. ThreatCrowd's public API returns non-200 for rate limiting and outages, and the library converts the status code into an Error Result before attempting to parse the body.

Solutions

Check resp.StatusCode and apply exponential backoff, especially for 429.
Disable or replace the threatcrowd source if the service is unavailable or deprecated.
Throttle requests well below the API's documented limits.

Defensive patterns

Strategy: retry

Try / catch

for r := range results {
  if r.Type == subscraping.Error {
    var code int
    if n, _ := fmt.Sscanf(r.Error.Error(), "unexpected status code: %d", &code); n == 1 && code == 429 {
      // back off significantly and retry later
    }
  }
}

Prevention

Heavily throttle ThreatCrowd requests; the public API rate-limits aggressively.
Disable the source when the service is down or deprecated.
Cache prior results to avoid repeat queries.

When it happens

Trigger: HTTP GET to the ThreatCrowd API returns any status other than 200 — most commonly 401/403 (key issues) or 429/5xx (rate limit or downtime).

Common situations: ThreatCrowd heavily rate-limits public queries; automated enumeration without throttling quickly triggers non-200 responses; the service has had extended outages/deprecations.

Related errors

AI-assisted analysis of projectdiscovery/subfinder@7a0b91f0fa (2026-09-06). Data as JSON: /api/errors/e2fc893d128d6112. Report an issue: GitHub.

Appendix: source

Thrown at pkg/subscraping/sources/threatcrowd/threatcrowd.go:69

		resp, err := session.Client.Do(req)
		if err != nil {
			results <- subscraping.Result{Source: s.Name(), Type: subscraping.Error, Error: err}
			s.errors++
			return
		}
		// This source issues a raw client.Do (bypassing the session's
		// httpRequestWrapper), so apply the response-body size cap explicitly
		// when configured (0 = unlimited).
		subscraping.LimitResponseBody(resp, session.MaxResponseBodySize)
		defer func() {
			if err := resp.Body.Close(); err != nil {
				results <- subscraping.Result{Source: s.Name(), Type: subscraping.Error, Error: err}
				s.errors++
			}
		}()

		if resp.StatusCode != http.StatusOK {
			results <- subscraping.Result{Source: s.Name(), Type: subscraping.Error, Error: fmt.Errorf("unexpected status code: %d", resp.StatusCode)}
			s.errors++
			return
		}

		body, err := io.ReadAll(resp.Body)
		if err != nil {
			results <- subscraping.Result{Source: s.Name(), Type: subscraping.Error, Error: err}
			s.errors++
			return
		}

		var tcResponse threatCrowdResponse
		if err := json.Unmarshal(body, &tcResponse); err != nil {
			results <- subscraping.Result{Source: s.Name(), Type: subscraping.Error, Error: err}
			s.errors++
			return
		}

View on GitHub (pinned to 7a0b91f0fa)