fluent/fluentd · error · Fluent::ConfigError

Duplicate key in <and>: #{e.key}

Error message

Duplicate key in <and>: #{e.key}

What it means

When folding nested <regexp> directives of an <and> section into the global AND-condition map, configure rejects a key that already exists in regexp_and_conditions (from regexpN lines, top-level <regexp> sections, or earlier <and> blocks' regexps) with Fluent::ConfigError 'Duplicate key in <and>'.

Source

Thrown at lib/fluent/plugin/filter_grep.rb:133

      @regexps.each do |e|
        raise Fluent::ConfigError, "Duplicate key: #{e.key}" if regexp_and_conditions.key?(e.key)
        regexp_and_conditions[e.key] = Expression.new(record_accessor_create(e.key), e.pattern)
      end

      if @excludes.size > 1
        log.info "Top level multiple <exclude> is interpreted as 'or' condition"
      end
      @excludes.each do |e|
        raise Fluent::ConfigError, "Duplicate key: #{e.key}" if exclude_or_conditions.key?(e.key)
        exclude_or_conditions[e.key] = Expression.new(record_accessor_create(e.key), e.pattern)
      end

      @and_conditions.each do |and_condition|
        if !and_condition.regexps.empty? && !and_condition.excludes.empty?
          raise Fluent::ConfigError, "Do not specify both <regexp> and <exclude> in <and>"
        end
        and_condition.regexps.each do |e|
          raise Fluent::ConfigError, "Duplicate key in <and>: #{e.key}" if regexp_and_conditions.key?(e.key)
          regexp_and_conditions[e.key] = Expression.new(record_accessor_create(e.key), e.pattern)
        end
        and_condition.excludes.each do |e|
          raise Fluent::ConfigError, "Duplicate key in <and>: #{e.key}" if exclude_and_conditions.key?(e.key)
          exclude_and_conditions[e.key] = Expression.new(record_accessor_create(e.key), e.pattern)
        end
      end

      @or_conditions.each do |or_condition|
        if !or_condition.regexps.empty? && !or_condition.excludes.empty?
          raise Fluent::ConfigError, "Do not specify both <regexp> and <exclude> in <or>"
        end
        or_condition.regexps.each do |e|
          raise Fluent::ConfigError, "Duplicate key in <or>: #{e.key}" if regexp_or_conditions.key?(e.key)
          regexp_or_conditions[e.key] = Expression.new(record_accessor_create(e.key), e.pattern)
        end
        or_condition.excludes.each do |e|
          raise Fluent::ConfigError, "Duplicate key in <or>: #{e.key}" if exclude_or_conditions.key?(e.key)

View on GitHub (pinned to dd45c6e18d)

Solutions

1. Use a different field, or consolidate the conditions on that field into one regex
2. If OR over the same field was intended, put the <regexp>s in an <or> section instead
3. Remove the duplicated directive identified by the key in the error message
4. Validate with fluentd --dry-run

Example fix

# before
<regexp>
  key message
  pattern /error/
</regexp>
<and>
  <regexp>
    key message
    pattern /fatal/
  </regexp>
</and>

# after
<regexp>
  key message
  pattern /(error|fatal)/
</regexp>

taken = Set.new
conf.sections(:regexp) { |s| taken << s['key'] }
(1..20).each { |i| conf["regexp#{i}"]&.split(/ /, 2)&.first&.then { |k| taken << k } }
conf.sections(:and).each do |a|
  a.sections(:regexp).each do |r|
    abort "duplicate regexp key in <and>: #{r['key']}" if taken.include?(r['key'])
    taken << r['key']
  end
end

begin
  filter.configure(conf)
rescue Fluent::ConfigError => e
  abort "#{e.message} — regexp AND-conditions share one global key namespace" if e.message.include?('Duplicate key in <and>')
  raise
end

When it happens

Trigger: An <and> section whose <regexp> key repeats a field already registered, e.g. top-level <regexp> key message ... plus <and> <regexp> key message ... </and>; or the same key in two different <and> blocks' regexp parts.

Common situations: Building complex and/or logic and reusing the same field in multiple places; assuming each <and> block has its own key namespace (it does not — regexp AND-conditions share one global map).

Related errors

• regexp#{i} contains a duplicated key, #{key}
• exclude#{i} contains a duplicated key, #{key}
• Duplicate key: #{e.key}
• regexp#{i} does not contain 2 parameters
• exclude#{i} does not contain 2 parameters

AI-assisted analysis of fluent/fluentd@dd45c6e18d (2026-08-21). Data as JSON: /api/errors/e190d2b5d1cd4809. Report an issue: GitHub.