What causes "Invalid regexp in #{key}: #{e}" in fluent/fluentd?

Reproduce the compile error standalone: ruby -e "Regexp.new(File.read? 'body', Regexp::MULTILINE)" or paste the body into a Ruby shell.

fluent/fluentd · error · Fluent::ConfigError

Invalid regexp in # : #

Error message

Invalid regexp in #{key}: #{e}

What it means

MultilineParser#check_format_regexp (lib/fluent/plugin/parser_multiline.rb:144) compiles an individual formatN or format_firstline value (delimiters '/' stripped) with Regexp::MULTILINE and wraps any RegexpError in Fluent::ConfigError. Unlike error 230 (which validates the joined regex), this validates each piece separately, pinpointing the exact key whose regex body is invalid Ruby regexp syntax.

Solutions

Reproduce the compile error standalone: ruby -e "Regexp.new(File.read? 'body', Regexp::MULTILINE)" or paste the body into a Ruby shell.
Escape embedded '/' inside the body as '\\/' or restructure the pattern.
Validate configs in CI with fluentd --dry-run.
Prefer simple anchor-heavy patterns and test them against sample log lines with ruby's Regexp before deploying.

Example fix

# before
format_firstline /^2021\/\d{2/  # unbalanced brace -> RegexpError
# after
format_firstline /^2021\/\d{2}\//

Defensive patterns

Strategy: validation

Validate before calling

# compile each formatN body before deploy
formats = ['/^\\d{4}-/', '(?<t>\\d{2}:\\d{2})']
formats.each { |body| Regexp.new(body, Regexp::MULTILINE) }

Prevention

Compile-test regexes in a Ruby shell (fluentd uses Ruby regexp, not PCRE).
Escape embedded '/' inside slash-delimited format bodies.
Beware templating layers (Helm/ERB) mangling backslashes; compare rendered output.

When it happens

Trigger: format3 /(?<a>\d+/ (unbalanced group); format_firstline /\d{4\/\d{2}/ with an unescaped '/' inside; quantifier mistakes like /a{,3}/ or /a**/; lookbehind constructs unsupported by fluentd's Ruby version.

Common situations: Hand-editing multiline formats for application logs; escaping regressions when configs pass through templating (Helm, ERB) that eats backslashes; regexes copied from PCRE-style tools (grep, Logstash) using syntax Ruby rejects.

Related errors

AI-assisted analysis of fluent/fluentd@dd45c6e18d (2026-08-21). Data as JSON: /api/errors/0fdc4e0b5d512056. Report an issue: GitHub.

Appendix: source

Thrown at lib/fluent/plugin/parser_multiline.rb:144

        }
      end

      def check_format_range(conf)
        invalid_formats = conf.keys.select { |k|
          m = k.match(/^format(\d+)$/)
          m ? !((1..FORMAT_MAX_NUM).include?(m[1].to_i)) : false
        }
        unless invalid_formats.empty?
          raise Fluent::ConfigError, "Invalid formatN found. N should be 1 - #{FORMAT_MAX_NUM}: " + invalid_formats.join(",")
        end
      end

      def check_format_regexp(format, key)
        if format[0] == '/' && format[-1] == '/'
          begin
            Regexp.new(format[1..-2], Regexp::MULTILINE)
          rescue => e
            raise Fluent::ConfigError, "Invalid regexp in #{key}: #{e}"
          end
        else
          raise Fluent::ConfigError, "format should be Regexp, need //, in #{key}: '#{format}'"
        end
      end
    end
  end
end

View on GitHub (pinned to dd45c6e18d)