we-promise/sure · warning · Provider::Tiingo::RateLimitError

Tiingo hourly request limit reached (#{new_count}/#{max_requ

Error message

Tiingo hourly request limit reached (#{new_count}/#{max_requests_per_hour})

What it means

This is a client-side quota gate, not a server error. throttle_request first applies the interval throttle from RateLimitable, then atomically increments a Redis counter keyed 'tiingo:requests:<epoch-hour>' (expires in 7200s). When the returned count is >= max_requests_per_hour (ENV TIINGO_MAX_REQUESTS_PER_HOUR, default MAX_REQUESTS_PER_HOUR = 1000), it raises RateLimitError. The raise happens BEFORE the HTTP request, so no API call is wasted and Tiingo never saw the blocked request.

Source

Thrown at app/models/provider/tiingo.rb:245

        faraday.request :json
        faraday.response :raise_error
        faraday.headers["Authorization"] = "Token #{api_key}"
        faraday.headers["Content-Type"] = "application/json"
      end
    end

    # Adds hourly request counter on top of the interval throttle from RateLimitable.
    def throttle_request
      super

      # Global per-hour request counter via cache (Redis).
      # Atomic increment-then-check avoids the TOCTOU of read-check-increment.
      hour_key = "tiingo:requests:#{Time.current.to_i / 3600}"
      new_count = Rails.cache.increment(hour_key, 1, expires_in: 7200.seconds).to_i

      if new_count >= max_requests_per_hour
        raise RateLimitError, "Tiingo hourly request limit reached (#{new_count}/#{max_requests_per_hour})"
      end
    end

    # Tracks unique symbols queried per month to stay within Tiingo's 500 symbols/month limit.
    # Uses atomic set-if-absent (Redis SETNX) to eliminate the read-then-write race
    # where two concurrent workers could both see the symbol as untracked and both
    # increment the counter.
    def track_symbol(symbol)
      symbol_key = "tiingo:symbol:#{Date.current.strftime('%Y-%m')}:#{symbol.upcase}"
      count_key  = "tiingo:symbol_count:#{Date.current.strftime('%Y-%m')}"

      # Atomic write-if-absent: returns false when the key already exists (Redis SETNX).
      # Only the first worker to claim this symbol will proceed to increment the counter.
      return unless Rails.cache.write(symbol_key, true, expires_in: 35.days, unless_exist: true)

      new_count = Rails.cache.increment(count_key, 1, expires_in: 35.days).to_i

      if new_count >= MAX_SYMBOLS_PER_MONTH

View on GitHub (pinned to e69894adb9)

Solutions

Wait for the hour window to roll over -- the key is 'tiingo:requests:#{Time.current.to_i / 3600}', so the counter resets at the next epoch-hour boundary
If your Tiingo plan allows more, raise TIINGO_MAX_REQUESTS_PER_HOUR in the environment
Reduce request volume: widen cache TTLs, batch date ranges into single fetch_security_prices calls instead of per-date fetch_security_price, reuse search currency caching

Example fix

# before (per-date lookups, one request each)
dates.each { |d| provider.fetch_security_price(symbol: sym, date: d) }

# after (one ranged request)
provider.fetch_security_prices(symbol: sym, start_date: dates.first, end_date: dates.last)

Defensive patterns

Strategy: retry

Validate before calling

# Check the shared counter before issuing a request
count = Rails.cache.read("tiingo:requests:#{Time.current.to_i / 3600}").to_i
raise Provider::Tiingo::RateLimitError, 'Local hourly budget spent' if count >= ENV.fetch('TIINGO_MAX_REQUESTS_PER_HOUR', 1000).to_i

Try / catch

begin
  provider.fetch_security_prices(...)
rescue Provider::Tiingo::RateLimitError
  SyncRetryJob.set(wait: until_next_epoch_hour).perform_later(...) # window resets on the hour
end

Prevention

Batch date ranges into single ranged calls instead of per-date requests
Set TIINGO_MAX_REQUESTS_PER_HOUR to the real plan limit, not the default
Prefer cached currency/search data to avoid repeat search calls

When it happens

Trigger: Any Tiingo call (search_securities, fetch_security_prices, fetch_security_price, healthy?) once the whole app (all processes share the Redis counter) has made ~1000 Tiingo requests within the current wall-clock hour. The message includes the live count over the cap, e.g. '(1000/1000)'.

Common situations: Bulk price-refresh jobs sweeping large portfolios; a retry storm elsewhere re-requesting prices; default 1000/hr cap left in place while the account is on a paid tier that allows more; cache misses forcing repeated search calls.

Related errors

AI-assisted analysis of we-promise/sure@e69894adb9 (2026-08-21). Data as JSON: /api/errors/e56db25806a98d0c. Report an issue: GitHub.