we-promise/sure · error · Provider::YahooFinance::Error

Invalid search response format: #

Error message

Invalid search response format: #{e.message}

What it means

Provider::YahooFinance::Error raised in search_securities when JSON.parse fails on the /v1/finance/search response. The endpoint answered with a non-JSON body (HTML block page, empty body, or changed content type), so parsing raised JSON::ParserError and got converted to a typed error.

Solutions

1. Capture and inspect the raw body when this fires to distinguish HTML blocks from truncation
2. Throttle searches (the provider paces at 0.5s; keep interactive autocomplete debounced too)
3. Retry after a delay or fall back to another security-search provider
4. Cache search results (the provider already caches 5 min) so repeat queries don't re-hit Yahoo

Example fix

// before
securities = provider.search_securities(params[:q])

// after
begin
  securities = provider.search_securities(params[:q])
rescue Provider::YahooFinance::Error => e
  raise unless e.message.start_with?("Invalid search response format")
  securities = fallback_provider.search_securities(params[:q])
end

return [] if query.to_s.strip.length < 2

begin
  provider.search_securities(query)
rescue Provider::YahooFinance::Error => e
  raise unless e.message.start_with?("Invalid search response format")
  sleep 5
  retry if (attempts += 1) < 2
  fallback_provider.search_securities(query)
end

When it happens

Trigger: The unauthenticated Yahoo search endpoint returning an HTML captcha/consent page or rate-limit page instead of JSON; an empty 204-style body; response intercepted by a proxy.

Common situations: Rapid symbol searches from a datacenter IP tripping Yahoo's bot defenses; Yahoo deploying breaking changes to the unofficial endpoint; intermittent CDN errors.

Understand the failure class

Background: "Invalid JSON response" and "Failed to parse response" errors: when an API answers 200 but the body isn't the JSON your library expected — this error's family across 28 libraries.

Related errors

• Invalid response format: #
• Failed to fetch exchange rates: #
• Failed to fetch security prices: #
• No chart data found for currency pair #
• No exchange rate found for #

AI-assisted analysis of we-promise/sure@e69894adb9 (2026-08-21). Data as JSON: /api/errors/f42a9ec1d43e5f9b. Report an issue: GitHub.

Appendix: source

Thrown at app/models/provider/yahoo_finance.rb:229

        securities = quotes.filter_map do |quote|
          mic = map_exchange_mic(quote["exchange"])

          Security.new(
            symbol: quote["symbol"],
            name: quote["longname"] || quote["shortname"] || quote["symbol"],
            logo_url: nil, # Yahoo search doesn't provide logos
            exchange_operating_mic: mic,
            country_code: ::Security::EXCHANGES.dig(mic, "country") || map_country_code(quote["exchDisp"])
          )
        end

        securities = deduplicate_dual_listings(securities) unless exchange_operating_mic.present?

        cache_result(cache_key, securities)
        securities
      end
    rescue JSON::ParserError => e
      raise Error, "Invalid search response format: #{e.message}"
    end
  end

  def fetch_security_info(symbol:, exchange_operating_mic:)
    with_provider_response do
      symbol = normalize_symbol(symbol, exchange_operating_mic)

      # quoteSummary endpoint requires cookie/crumb authentication
      throttle_request
      cookie, crumb = fetch_cookie_and_crumb

      response = authenticated_client(cookie).get("#{base_url}/v10/finance/quoteSummary/#{symbol}") do |req|
        req.params["modules"] = "assetProfile,price,quoteType"
        req.params["crumb"] = crumb
      end

      data = JSON.parse(response.body)

View on GitHub (pinned to e69894adb9)