fluent/fluentd · error · Fluent::ConfigError

#{e}. Service must have `host` and `port`

Error message

#{e}. Service must have `host` and `port`

What it means

SdFileServiceDiscovery#fetch_server_info (lib/fluent/plugin/sd_file.rb:138) maps each YAML/JSON service entry to a Service object using Hash#fetch for 'host' and 'port' — which raise KeyError when absent — and re-wraps that as Fluent::ConfigError. Entries may omit every other field (name, weight, standby, username...) but host and port are mandatory. The error names the missing key in its message.

Source

Thrown at lib/fluent/plugin/sd_file.rb:138

          rescue => e
            raise Fluent::ConfigError, "sd_file: path=#{@path} couldn't open #{e}"
          end

        parser.call(config_data).map do |s|
          Service.new(
            :file,
            s.fetch('host'),
            s.fetch('port'),
            s['name'],
            s.fetch('weight', DEFAULT_WEIGHT),
            s['standby'],
            s['username'],
            s['password'],
            s['shared_key'],
          )
        end
      rescue KeyError => e
        raise Fluent::ConfigError, "#{e}. Service must have `host` and `port`"
      end

      class StatWatcher < Coolio::StatWatcher
        def initialize(path, log, &callback)
          @path = path
          @log = log
          @callback = callback
          super(@path)
        end

        def on_change(prev_stat, cur_stat)
          @callback.call(prev_stat, cur_stat)
        rescue => e
          @log.error(e)
        end
      end
    end
  end

View on GitHub (pinned to dd45c6e18d)

Solutions

1. Add host and port to every entry: - host: 192.0.2.1\\n port: 24224.
2. Validate the file shape before restart (yaml parse + each entry has_key host/port).
3. Regenerate the file from source data with the correct field mapping.
4. Use fluentd --dry-run to fail fast at deploy time instead of at runtime reload.

Example fix

# before
# /etc/fluent/services.yaml
- name: node1
- name: node2
  host: 192.0.2.2
  port: 24224
# after
- name: node1
  host: 192.0.2.1
  port: 24224
- name: node2
  host: 192.0.2.2
  port: 24224

require 'yaml'
YAML.load_file('/etc/fluent/services.yaml').each do |s|
  abort "entry missing host/port: #{s}" unless s['host'] && s.key?('port')
end

def complete_service_entry?(s)
  s.is_a?(Hash) && s.key?('host') && s.key?('port')
end

When it happens

Trigger: A services.yaml entry like '- name: node1' (no host/port); port specified as a string key 'port:' with empty value yielding nil is accepted, but truly absent keys raise; JSON entries where host is nested one level deeper than expected; entries using 'address' instead of 'host'.

Common situations: Hand-editing service discovery YAML and forgetting the port; generating the file from a CMDB export with different field names; truncated YAML from a failed template render; mixing indentation so entries land under a comments/metadata key.

Related errors

• sd_file: path=#{@path} not found
• sd_file: path=#{@path} couldn't open #{e}
• #{@key_name} does not exist
• unknown value conversion for key:'#{field_name}', type:'#{ty
• Invalid regexp '#{formats}': #{e}

AI-assisted analysis of fluent/fluentd@dd45c6e18d (2026-08-21). Data as JSON: /api/errors/f06754e28f45712a. Report an issue: GitHub.