What causes "Receiving missing id data: #{data}" in fluent/fluentd?

If the warning is rare and counters otherwise work, treat it as noise from a duplicated/stale reply and ignore it.

Receiving missing id data: #{data}

Error message

Receiving missing id data: #{data}

What it means

Fluent::Counter::Client correlates each request with a Future stored in @responses keyed by a generated id; on_message resolves the Future when a response with a matching data['id'] arrives. This warning fires when the counter server sends a response whose id matches no pending request — the reply is orphaned (already answered, duplicated, or stale after reconnect) and is discarded.

Source

Thrown at lib/fluent/counter/client.rb:165

          Thread.start do
            yield res.get
          end
        else
          res
        end
      end

      private

      def exist_scope!
        raise 'Call `establish` method to get a `scope` before calling this method' unless @scope
      end

      def on_message(data)
        if response = @responses.delete(data['id'])
          response.set(data)
        else
          @log.warn("Receiving missing id data: #{data}")
        end
      end

      def send_request(method, scope, params, opt = {})
        id = generate_id
        res = Future.new(@loop, @loop_mutex)
        @responses[id] = res # set a response value to this future object at `on_message`
        request = build_request(method, id, scope, params, opt)
        @log.debug(request)
        @conn.send_data request
        res
      end

      def build_request(method, id, scope = nil, params = nil, options = nil)
        r = { id: id, method: method }
        r[:scope] = scope if scope
        r[:params] = params if params
        r[:options] = options if options

View on GitHub (pinned to dd45c6e18d)

Solutions

If the warning is rare and counters otherwise work, treat it as noise from a duplicated/stale reply and ignore it.
Verify only one Counter::Client instance consumes the socket (no two clients sharing host:port).
Ensure the counter server responds exactly once per request and echoes the request id unchanged.
Check for client/server Fluentd version skew on the counter plugin API and align versions.

Defensive patterns

Strategy: validation

Type guard

# Guard before acting on a counter-server response (mirrors the library's own check)
def handle_response(data, pending_ids)
  return unless data.is_a?(Hash) && pending_ids.include?(data['id'])
  # ...process matched response...
end

Prevention

Run exactly one Counter::Client per process/endpoint; never share the socket.
Ensure custom counter servers echo request ids verbatim and reply exactly once.
Keep client and server Fluentd versions aligned when using the counter API.

When it happens

Trigger: The counter server sends two responses for one request; a response arrives after the caller timed out/stopped waiting and the entry was deleted; client reconnected to a server that replays old replies; a custom/test counter server emitting unsolicited messages; version mismatch in the counter protocol between client and server.

Common situations: Using the fluent-counter CLI or the plugin counter API against a custom counter server; flaky loopback connections causing duplicate delivery; integration tests with mock servers that do not honor request ids.

Related errors

<counter_client> is required in <system>

AI-assisted analysis of fluent/fluentd@dd45c6e18d (2026-08-21). Data as JSON: /api/errors/db581a2ff5f1f8f5. Report an issue: GitHub.