we-promise/sure · error · Provider::YahooFinance::Error

Invalid search response format: #{e.message}

Error message

Invalid search response format: #{e.message}

What it means

Provider::YahooFinance::Error raised in search_securities when JSON.parse fails on the /v1/finance/search response. The endpoint answered with a non-JSON body (HTML block page, empty body, or changed content type), so parsing raised JSON::ParserError and got converted to a typed error.

Source

Thrown at app/models/provider/yahoo_finance.rb:229

        securities = quotes.filter_map do |quote|
          mic = map_exchange_mic(quote["exchange"])

          Security.new(
            symbol: quote["symbol"],
            name: quote["longname"] || quote["shortname"] || quote["symbol"],
            logo_url: nil, # Yahoo search doesn't provide logos
            exchange_operating_mic: mic,
            country_code: ::Security::EXCHANGES.dig(mic, "country") || map_country_code(quote["exchDisp"])
          )
        end

        securities = deduplicate_dual_listings(securities) unless exchange_operating_mic.present?

        cache_result(cache_key, securities)
        securities
      end
    rescue JSON::ParserError => e
      raise Error, "Invalid search response format: #{e.message}"
    end
  end

  def fetch_security_info(symbol:, exchange_operating_mic:)
    with_provider_response do
      symbol = normalize_symbol(symbol, exchange_operating_mic)

      # quoteSummary endpoint requires cookie/crumb authentication
      throttle_request
      cookie, crumb = fetch_cookie_and_crumb

      response = authenticated_client(cookie).get("#{base_url}/v10/finance/quoteSummary/#{symbol}") do |req|
        req.params["modules"] = "assetProfile,price,quoteType"
        req.params["crumb"] = crumb
      end

      data = JSON.parse(response.body)

View on GitHub (pinned to e69894adb9)

Solutions

1. Capture and inspect the raw body when this fires to distinguish HTML blocks from truncation
2. Throttle searches (the provider paces at 0.5s; keep interactive autocomplete debounced too)
3. Retry after a delay or fall back to another security-search provider
4. Cache search results (the provider already caches 5 min) so repeat queries don't re-hit Yahoo

Example fix

// before
securities = provider.search_securities(params[:q])

// after
begin
  securities = provider.search_securities(params[:q])
rescue Provider::YahooFinance::Error => e
  raise unless e.message.start_with?("Invalid search response format")
  securities = fallback_provider.search_securities(params[:q])
end

return [] if query.to_s.strip.length < 2

begin
  provider.search_securities(query)
rescue Provider::YahooFinance::Error => e
  raise unless e.message.start_with?("Invalid search response format")
  sleep 5
  retry if (attempts += 1) < 2
  fallback_provider.search_securities(query)
end

When it happens

Trigger: The unauthenticated Yahoo search endpoint returning an HTML captcha/consent page or rate-limit page instead of JSON; an empty 204-style body; response intercepted by a proxy.

Common situations: Rapid symbol searches from a datacenter IP tripping Yahoo's bot defenses; Yahoo deploying breaking changes to the unofficial endpoint; intermittent CDN errors.

Related errors

• Invalid response format: #{e.message}
• Yahoo Finance rate limit exceeded
• Failed to fetch exchange rates: #{rates_response.error.messa
• No exchange rate found for #{from}/#{to} on or before #{date
• No chart data found for currency pair #{from}/#{to}

AI-assisted analysis of we-promise/sure@e69894adb9 (2026-08-21). Data as JSON: /api/errors/f42a9ec1d43e5f9b. Report an issue: GitHub.