we-promise/sure · critical · Provider::YahooFinance::AuthenticationError

Yahoo Finance authentication failed after crumb refresh

Error message

Yahoo Finance authentication failed after crumb refresh

What it means

Provider::YahooFinance::AuthenticationError raised in fetch_security_info when the quoteSummary endpoint answers 'Unauthorized' even after the client cleared its cached cookie/crumb pair and fetched a fresh one. Yahoo's unofficial API requires a session cookie plus a crumb token; two consecutive Unauthorized responses mean the cookie/crumb flow itself is broken (blocked, expired, or rejected).

Source

Thrown at app/models/provider/yahoo_finance.rb:259

      response = authenticated_client(cookie).get("#{base_url}/v10/finance/quoteSummary/#{symbol}") do |req|
        req.params["modules"] = "assetProfile,price,quoteType"
        req.params["crumb"] = crumb
      end

      data = JSON.parse(response.body)

      # Check for auth errors in response body
      if data.dig("quoteSummary", "error", "code") == "Unauthorized"
        # Clear cached crumb and retry once
        clear_crumb_cache
        cookie, crumb = fetch_cookie_and_crumb
        response = authenticated_client(cookie).get("#{base_url}/v10/finance/quoteSummary/#{symbol}") do |req|
          req.params["modules"] = "assetProfile,price,quoteType"
          req.params["crumb"] = crumb
        end
        data = JSON.parse(response.body)
        if data.dig("quoteSummary", "error", "code") == "Unauthorized"
          raise AuthenticationError, "Yahoo Finance authentication failed after crumb refresh"
        end
      end

      result = data.dig("quoteSummary", "result", 0)

      raise Error, "No security info found for #{symbol}" unless result

      asset_profile = result["assetProfile"] || {}
      price_info = result["price"] || {}
      quote_type = result["quoteType"] || {}

      security_info = SecurityInfo.new(
        symbol: symbol,
        name: price_info["longName"] || price_info["shortName"] || quote_type["longName"] || quote_type["shortName"],
        links: asset_profile["website"],
        logo_url: nil, # Yahoo doesn't provide reliable logo URLs
        description: asset_profile["longBusinessSummary"],
        kind: map_security_type(quote_type["quoteType"]),

View on GitHub (pinned to e69894adb9)

Solutions

Wait and retry later — Yahoo blocks are usually temporary; check provider.health_status for the tracked rate_limited/unavailable state
Test fetch_cookie_and_crumb in isolation (console) to see whether crumb issuance returns a valid token or an error/HTML
Rotate the egress IP or update the USER_AGENTS pool to current browser versions when Yahoo starts rejecting old fingerprints
Clear the yahoo_finance cookie/crumb cache keys if a poisoned cookie is cached; fall back to another securities provider for info lookups

Example fix

// before
info = provider.fetch_security_info(symbol: "AAPL", exchange_operating_mic: "XNAS")

// after
begin
  info = provider.fetch_security_info(symbol: "AAPL", exchange_operating_mic: "XNAS")
rescue Provider::YahooFinance::AuthenticationError
  if provider.health_status == :rate_limited
    RetryableSyncJob.perform_later(wait: 30.minutes)
  else
    info = fallback_provider.fetch_security_info(symbol: "AAPL", exchange_operating_mic: "XNAS")
  end
end

Defensive patterns

Strategy: try-catch

Try / catch

begin
  provider.fetch_security_info(symbol:, exchange_operating_mic:)
rescue Provider::YahooFinance::AuthenticationError
  if provider.health_status == :rate_limited
    RetryableJob.perform_later(wait: 30.minutes)
  else
    fallback_provider.fetch_security_info(symbol:, exchange_operating_mic:)
  end
end

Prevention

Check provider.health_status before hitting crumb-authenticated endpoints; it already tracks Yahoo blocks
Keep the USER_AGENTS pool current (the codebase refreshes it when Yahoo tightens checks)
Cache successful quoteSummary lookups to minimize crumb-protected calls
Clear poisoned cookie/crumb cache entries when blocks are suspected

When it happens

Trigger: The /v10/finance/quoteSummary call returning error.code 'Unauthorized' with a stale crumb, then again after clear_crumb_cache + fetch_cookie_and_crumb — e.g. the consent page served instead of a cookie, the crumb endpoint returning 'too many requests' crumbs (see INVALID_CRUMBS), or Yahoo blocking the client's IP/User-Agent from crumb issuance.

Common situations: Yahoo tightening anti-bot measures so cookie+crumb acquisition silently fails; datacenter IPs flagged; the cached cookie store (shared via Rails.cache) poisoned with a cookie another blocked process obtained; outdated User-Agent pool after a Yahoo crackdown (the codebase updates USER_AGENTS periodically for exactly this).

Understand the failure class

Authentication and authorization failures — expired tokens, bad credentials, and missing scopes.

Related errors

AI-assisted analysis of we-promise/sure@e69894adb9 (2026-08-21). Data as JSON: /api/errors/f658ae2e3e010ce0. Report an issue: GitHub.