fluent/fluentd · error · Fluent::ConfigError

Invalid formatN found. N should be 1 - #{FORMAT_MAX_NUM}: #{

Error message

Invalid formatN found. N should be 1 - #{FORMAT_MAX_NUM}: #{invalid_formats.join(",")}

What it means

MultilineParser#check_format_range (lib/fluent/plugin/parser_multiline.rb:135) scans every config key matching /\Aformat(\d+)\z/ and raises Fluent::ConfigError listing any formatN whose N falls outside 1..FORMAT_MAX_NUM (20, parser_multiline.rb:30). The parser only iterates a fixed 1..20 range, so format0, format21, or format999 would otherwise be silently ignored — this error makes the omission loud. Raised at configure time.

Source

Thrown at lib/fluent/plugin/parser_multiline.rb:135

          format = conf["format#{i}"]
          if (i > 1) && prev_format.nil? && !format.nil?
            raise Fluent::ConfigError, "Jump of format index found. format#{i - 1} is missing."
          end
          prev_format = format
          next if format.nil?

          check_format_regexp(format, "format#{i}")
          format
        }
      end

      def check_format_range(conf)
        invalid_formats = conf.keys.select { |k|
          m = k.match(/^format(\d+)$/)
          m ? !((1..FORMAT_MAX_NUM).include?(m[1].to_i)) : false
        }
        unless invalid_formats.empty?
          raise Fluent::ConfigError, "Invalid formatN found. N should be 1 - #{FORMAT_MAX_NUM}: " + invalid_formats.join(",")
        end
      end

      def check_format_regexp(format, key)
        if format[0] == '/' && format[-1] == '/'
          begin
            Regexp.new(format[1..-2], Regexp::MULTILINE)
          rescue => e
            raise Fluent::ConfigError, "Invalid regexp in #{key}: #{e}"
          end
        else
          raise Fluent::ConfigError, "format should be Regexp, need //, in #{key}: '#{format}'"
        end
      end
    end
  end
end

View on GitHub (pinned to dd45c6e18d)

Solutions

1. Keep N within 1..20 and merge overflow segments into format20's pattern.
2. If you need more granularity, switch to a single combined regex with multiple named captures instead of many formatN keys.
3. Fix 0-based numbering: rename format0 to format1 and shift the rest.

Example fix

# before
format20 /tail/
format21 /more-tail/   # out of range -> ConfigError
# after
format20 /tail|more-tail/

conf.keys.each do |k|
  if (m = k.match(/\Aformat(\d+)\z/))
    n = m[1].to_i
    abort "#{k} out of range" unless (1..20).cover?(n)
  end
end

When it happens

Trigger: format21 configured for a very long multi-regex chain; format0 used as a 'prefix' section; typos like format101; configs generated programmatically that assume unbounded N.

Common situations: Extremely detailed multiline formats (nested stack traces) exceeding 20 segments; users assuming 0-based numbering; pasted configs from sources that numbered from a different base.

Related errors

• Invalid regexp '#{formats}': #{e}
• Jump of format index found. format#{i - 1} is missing.
• Invalid regexp in #{key}: #{e}
• format should be Regexp, need //, in #{key}: '#{format}'
• unknown value conversion for key:'#{field_name}', type:'#{ty

AI-assisted analysis of fluent/fluentd@dd45c6e18d (2026-08-21). Data as JSON: /api/errors/904e8c41648d00e4. Report an issue: GitHub.