fluent/fluentd · error · Fluent::ConfigError

`renew_record` must be true to use `keep_keys`

Error message

`renew_record` must be true to use `keep_keys`

What it means

record_transformer rebuilds a record from <record> placeholders. keep_keys lists keys from the original record to carry over, which only has meaning when renew_record true (the new record replaces the old one); without renewal the original fields are already kept. At configure time, specifying keep_keys while renew_record is false (its default) raises Fluent::ConfigError '`renew_record` must be true to use `keep_keys`', so the config never loads.

Source

Thrown at lib/fluent/plugin/filter_record_transformer.rb:59

    desc 'When set to true, the full Ruby syntax is enabled in the ${...} expression.'
    config_param :enable_ruby, :bool, default: false
    desc 'Use original value type.'
    config_param :auto_typecast, :bool, default: true

    def configure(conf)
      super

      map = {}
      # <record></record> directive
      conf.elements.select { |element| element.name == 'record' }.each do |element|
        element.each_pair do |k, v|
          element.has_key?(k) # to suppress unread configuration warning
          map[k] = parse_value(v)
        end
      end

      if @keep_keys
        raise Fluent::ConfigError, "`renew_record` must be true to use `keep_keys`" unless @renew_record
      end

      @key_deleters = if @remove_keys
                        @remove_keys.map { |k| record_accessor_create(k) }
                      end

      placeholder_expander_params = {
        log: log,
        auto_typecast: @auto_typecast,
      }
      @placeholder_expander =
        if @enable_ruby
          # require utilities which would be used in ruby placeholders
          require 'pathname'
          require 'uri'
          require 'cgi/escape'
          RubyPlaceholderExpander.new(placeholder_expander_params)
        else

View on GitHub (pinned to dd45c6e18d)

Solutions

1. Add renew_record true when you really want a fresh record keeping only the listed keys.
2. Or drop keep_keys and use remove_keys [...] to strip fields from the original record (no renewal needed).
3. Validate with fluentd --dry-run before deploying.

Example fix

# before
<filter app.**>
  @type record_transformer
  keep_keys ["time","host"]
  <record>
    hostname ${host}
  </record>
</filter>
# after
<filter app.**>
  @type record_transformer
  renew_record true
  keep_keys ["time","host"]
  <record>
    hostname ${host}
  </record>
</filter>

# config lint: keep_keys requires renew_record
if conf =~ /keep_keys/ && conf !~ /renew_record\s+true/
  abort 'record_transformer: keep_keys without renew_record true'
end
# authoritative check: fluentd --dry-run -c /etc/fluent/fluent.conf

begin
  Fluent::Plugin.new_filter('record_transformer').configure(conf)
rescue Fluent::ConfigError => e
  abort "record_transformer config rejected: #{e.message}"
end

When it happens

Trigger: A record_transformer block containing keep_keys [...] without renew_record true.

Common situations: Operators adding keep_keys to selectively preserve fields without realizing it implies full record renewal; configs migrated from examples that omitted the renew_record line; intention was actually remove_keys (which works without renewal).

Related errors

• Duplicate key in <or>: #{e.key}
• empty value is specified in fields parameter
• 'tag' or 'tag_key' option is required on exec input
• e.message
• unknown_key

AI-assisted analysis of fluent/fluentd@dd45c6e18d (2026-08-21). Data as JSON: /api/errors/e877acc6e16a891e. Report an issue: GitHub.