fluent/fluentd · warning · Fluent::Counter::InvalidParams

invalid_params

invalid_params

Error message

#{key} already exists in counter

What it means

Fluentd's Windows supervisor runs a dedicated signal thread (install_windows_event_handler, lib/fluent/supervisor.rb:290) that waits on named Win32 events (STOP/HUP/USR1/USR2/CONT, plus 5 more when --signame is set) via Win32::Ipc#wait_any, which wraps the Win32 WaitForMultipleObjects API. The expected contract is a 1-based index of the signaled event; this warning fires when wait_any returns anything outside 1..events.length, i.e. 0 (WAIT_TIMEOUT, unexpected with the infinite 0xFFFFFFFF timeout) or a WAIT_FAILED-style error value. It means the wait call itself misbehaved (invalid/closed handle, gem version with different return semantics, duplicate event names), so no reliable signal can be dispatched.

Source

Thrown at lib/fluent/counter/store.rb:76

        end

        # storage_local calls PluginId#plugin_root_dir
        def plugin_root_dir
          nil
        end
      end

      def start
        @storage.load
      end

      def stop
        @storage.save
      end

      def init(key, data, ignore: false)
        ret = if v = get(key)
                raise InvalidParams.new("#{key} already exists in counter") unless ignore
                v
              else
                @storage.put(key, build_value(data))
              end

        build_response(ret)
      end

      def get(key, raise_error: false, raw: false)
        ret = if raise_error
                @storage.get(key) or raise UnknownKey.new("`#{key}` doesn't exist in counter")
              else
                @storage.get(key)
              end
        if raw
          ret
        else
          ret && build_response(ret)

View on GitHub (pinned to dd45c6e18d)

Solutions

1. Check gem compatibility: run fluent-gem list win32-event win32-ipc and align them with the versions pinned in fluentd's Gemfile.lock for your release; reinstall with fluent-gem install -v <pinned version> if they diverge.
2. Ensure the --signame (if set) is unique per fluentd instance on the machine and no leftover fluentd processes hold the same named events (tasklist | findstr fluentd, then stop duplicates).
3. Upgrade fluentd to the latest 1.x patch release, which contains fixes to the Windows event handling thread.
4. Reproduce with the ipc_idx value from the log: 0 means timeout/WAIT_FAILED surfaced as 0, a large value points to a handle-count/validity problem - use it to decide between gem downgrade and handle-investigation.
5. Restart the service after a clean shutdown (all fluentd child processes exited) so the named events are recreated fresh.

Example fix

// before (lib/fluent/supervisor.rb:317-327)
ipc_idx = ipc.wait_any(events.map {|e| e[:win32_event]}, infinite)
event_idx = ipc_idx - 1
if event_idx >= 0 && event_idx < events.length
  $log.debug("Got Win32 event \"#{events[event_idx][:win32_event].name}\"")
else
  $log.warn("Unexpected return value of Win32::Ipc#wait_any: #{ipc_idx}")
end
case events[event_idx][:action] # events[-1] silently dispatches the LAST event!

// after
ipc_idx = ipc.wait_any(events.map {|e| e[:win32_event]}, infinite)
event_idx = ipc_idx - 1
unless event_idx >= 0 && event_idx < events.length
  $log.warn("Unexpected return value of Win32::Ipc#wait_any: #{ipc_idx}")
  next # never dispatch on events[-1]; re-enter the wait loop
end
case events[event_idx][:action]

# before entering the wait loop (fork of install_windows_event_handler)
MAX_WAIT_OBJECTS = 64 # WaitForMultipleObjects hard limit
unless events.all? { |e| e[:win32_event].is_a?(Win32::Event) }
  raise ArgumentError, 'every wait object must be a Win32::Event'
end
if events.size > MAX_WAIT_OBJECTS
  raise ArgumentError, "too many events for WaitForMultipleObjects: #{events.size}"
end

# Ruby guard for the wait_any contract (1-based index in range)
def valid_wait_index?(ipc_idx, events)
  ipc_idx.is_a?(Integer) && (1..events.length).cover?(ipc_idx)
end

ipc_idx = ipc.wait_any(handles, 0xFFFFFFFF)
event_idx = ipc_idx - 1
next unless valid_wait_index?(ipc_idx, events) # events[-1] must never dispatch

begin
  ipc_idx = ipc.wait_any(events.map { |e| e[:win32_event] }, 0xFFFFFFFF)
rescue => e
  $log.error("Win32::Ipc#wait_any raised: #{e.class}: #{e.message}")
  next # re-enter loop; do not index events with a bogus value
end

When it happens

Trigger: Calling Win32::Ipc#wait_any with the 6 (or 11 with @signame) named events built in install_windows_event_handler and receiving 0, a negative value, or a value > events.length. Concretely: (a) the win32-event/win32-ipc gem version returns 0-based or nil results instead of 1-based; (b) a handle was closed while the wait was pending (e.g. stop_windows_event_thread closed the events in the ensure block during shutdown, leaving stale handles); (c) two fluentd instances share the same --signame so the named events collide and the wait fails; (d) WaitForMultipleObjects returns WAIT_FAILED for an invalid handle.

Common situations: Running fluentd/td-agent as a Windows service after a gem upgrade that changed win32-event semantics; multiple fluentd processes configured with the same signame; shutdown/restart races where the event objects are closed while the signal thread is still inside wait_any; environments where the bundled gem set diverges from fluentd's Gemfile.lock.

Related errors

• unknown_key
• Unknown #{@kind} plugin '#{type}'. Run 'gem search -rd fluen
• Missing '@type' parameter on <#{e.name}> directive
• '#{capability}' is not valid capability. Valid Capabilities
• Input must be a map (got #{record.class})

AI-assisted analysis of fluent/fluentd@dd45c6e18d (2026-08-21). Data as JSON: /api/errors/6bb3b3e1d18a883c. Report an issue: GitHub.