vxcontrol/pentagi · warning · RetryableError

failed to do request: %v

Error message

failed to do request: %v

What it means

client.Do(req) failed at the transport level — DNS failure, connection refused/refused proxy, TLS handshake, or context cancellation. Because transport errors are often transient, firecrawl returns a RetryableError (retryAfter=0).

Source

Thrown at backend/pkg/tools/searchers/firecrawl.go:167

		},
	}
	reqBody, err := json.Marshal(reqPayload)
	if err != nil {
		return "", Fatal(fmt.Errorf("failed to marshal request body: %v", err))
	}

	req, err := http.NewRequest(http.MethodPost, f.searchURL(), bytes.NewBuffer(reqBody))
	if err != nil {
		return "", Fatal(fmt.Errorf("failed to build request: %v", err))
	}

	req = req.WithContext(ctx)
	req.Header.Set("Content-Type", "application/json")
	req.Header.Set("Authorization", "Bearer "+f.apiKey())

	resp, err := client.Do(req)
	if err != nil {
		return "", Retryable(fmt.Errorf("failed to do request: %v", err), 0)
	}
	defer resp.Body.Close()

	return f.parseHTTPResponse(ctx, query, resp)
}

func (f *firecrawl) parseHTTPResponse(ctx context.Context, query string, resp *http.Response) (string, error) {
	switch resp.StatusCode {
	case http.StatusOK:
		var respBody firecrawlSearchResult
		if err := json.NewDecoder(resp.Body).Decode(&respBody); err != nil {
			return "", Fatal(fmt.Errorf("failed to decode response body: %v", err))
		}
		if !respBody.Success {
			if respBody.Error != "" {
				return "", Fatal(fmt.Errorf("request failed: %s", respBody.Error))
			}
			return "", Fatal(fmt.Errorf("request failed"))

View on GitHub (pinned to ea665308ba)

Solutions

1. Retry via the orchestrator — the error is typed retryable.
2. Verify network egress from the container: curl -v https://<firecrawl-url>.
3. Check DNS/proxy settings and firewall rules for the deployment.
4. If self-hosting Firecrawl, confirm the service is up and its TLS cert valid.
5. Increase the request/flow timeout if wrapped cause is context deadline exceeded.

Example fix

// before: no egress test, silent misconfig
FIRECRAWL_URL=https://firecrawl.internal:9443
// after: verify first
curl -v https://firecrawl.internal:9443/  # then fix port/service

// reachability preflight
ctx, cancel := context.WithTimeout(context.Background(), 3*time.Second)
defer cancel()
req, _ := http.NewRequestWithContext(ctx, http.MethodHead, cfg.ServerURL, nil)
if _, err := http.DefaultClient.Do(req); err != nil { return fmt.Errorf("engine unreachable: %w", err) }

func isNetworkError(err error) bool {
  var ne net.Error
  return errors.As(err, &ne) || errors.Is(err, context.DeadlineExceeded) || errors.Is(err, syscall.ECONNREFUSED)
}

err := searcher.Handle(ctx, q)
if err != nil && searchers.IsRetryable(err) {
  select {
  case <-time.After(backoff):
  case <-ctx.Done():
  }
  retry()
}

When it happens

Trigger: Network outage or wrong host/port for the Firecrawl endpoint, proxy down, ctx cancelled (deadline) mid-request while calling search.

Common situations: Container without egress to api.firecrawl.dev, DNS misconfig, firewall rules, expired TLS cert on a self-hosted Firecrawl, request deadline exceeded.

Related errors

• temporal window search failed: %w
• entity relationships search failed: %w
• google search failed: %w
• failed to do request: %w
• request to Sploitus failed: %w

AI-assisted analysis of vxcontrol/pentagi@ea665308ba (2026-09-01). Data as JSON: /api/errors/1498f57380983c9c. Report an issue: GitHub.