puppetlabs/puppet · error · Puppet::Util::RetryAction::RetryException::RetriesExceeded

%{retries} exceeded

Error message

%{retries} exceeded

What it means

Puppet::Util::RetryAction.retry_action (retry_action.rb:33) re-runs the block with exponential backoff (sleep ((2**failures)-1)*0.1) whenever an exception in retry_exceptions (default [StandardError]) is raised. Once the failure count reaches options[:retries], it raises RetryException::RetriesExceeded with '<retries> exceeded', carrying the original exception's backtrace. Note RetryException inherits from Exception, NOT StandardError, so a bare 'rescue => e' will not catch it.

Source

Thrown at lib/puppet/util/retry_action.rb:33

  def self.retry_action(options = {})
    # Retry actions for a specified amount of time. This method will allow the final
    # retry to complete even if that extends beyond the timeout period.
    unless block_given?
      raise RetryException::NoBlockGiven
    end

    retries = options[:retries]
    if retries.nil?
      raise RetryException::NoRetriesGiven
    end

    retry_exceptions = options[:retry_exceptions] || [StandardError]
    failures = 0
    begin
      yield
    rescue *retry_exceptions => e
      if failures >= retries
        raise RetryException::RetriesExceeded, _("%{retries} exceeded") % { retries: retries }, e.backtrace
      end

      Puppet.info(_("Caught exception %{klass}:%{error} retrying") % { klass: e.class, error: e })

      failures += 1

      # Increase the amount of time that we sleep after every
      # failed retry attempt.
      sleep(((2**failures) - 1) * 0.1)

      retry
    end
  end
end

View on GitHub (pinned to e227c27540)

Solutions

1. Fix or verify the underlying cause first: the original exception is logged via Puppet.info before the final raise and its backtrace is attached.
2. Raise options[:retries] to cover realistic startup/timeout windows (backoff grows roughly exponentially from 0.1s).
3. Narrow :retry_exceptions to the specific errors worth retrying (e.g., [Errno::ECONNREFUSED, Timeout::Error]) so unrelated bugs fail fast.
4. Catch Puppet::Util::RetryAction::RetryException::RetriesExceeded explicitly (it is an Exception, not StandardError) and escalate with context.

Example fix

// before
Puppet::Util::RetryAction.retry_action(retries: 2) { client.create } # raises '2 exceeded' on flaky API

// after
begin
  Puppet::Util::RetryAction.retry_action(retries: 6, retry_exceptions: [Errno::ECONNREFUSED, Timeout::Error]) { client.create }
rescue Puppet::Util::RetryAction::RetryException::RetriesExceeded => e
  raise "service still unreachable after backoff: #{e.message}"
end

begin
  Puppet::Util::RetryAction.retry_action(retries: retries, retry_exceptions: retryable) { yield }
rescue Puppet::Util::RetryAction::RetryException::RetriesExceeded => e
  raise "gave up after #{retries} retries: #{e.message}" # inherits Exception; a bare rescue will NOT catch it
end

When it happens

Trigger: retry_action(retries: 3) { flaky_network_call } where the call keeps raising StandardError subclasses; retries: 0 combined with any single failure raises immediately; wrapping service/DB start loops whose downtime exceeds 3-4 backoff cycles (0.1s, 0.3s, 0.7s, 1.5s...).

Common situations: Waiting for a database or HTTP service to become reachable during provisioning; retrying flaky external APIs with a too-small :retries; retry-exceptions misconfigured so an unexpected error class is counted.

Related errors

• Timeout waiting for exclusive lock on %{path}
• Too many HTTP retries for %{addr}
• Timed out waiting for #{@resource[:name]} to transition stat
• Timed out waiting for #{@resource[:name]} to transition stat
• The timeout must be a number.

AI-assisted analysis of puppetlabs/puppet@e227c27540 (2026-08-21). Data as JSON: /api/errors/88528fc14477e238. Report an issue: GitHub.