{"record":{"id":"6cfa4f9196bcbb29","repo":"fluent/fluentd","slug":"parse-failed-e-message","errorCode":null,"errorMessage":"parse failed #{e.message}","messagePattern":"parse failed #(.+?)","errorType":"exception","errorClass":"Fluent::Plugin::Parser::ParserError","httpStatus":null,"severity":"warning","filePath":"lib/fluent/plugin/filter_parser.rb","lineNumber":109,"sourceCode":"          else\n            router.emit_error_event(tag, time, record, Fluent::Plugin::Parser::ParserError.new(\"pattern not matched with data '#{raw_value}'\")) if @emit_invalid_record_to_error\n            next unless @reserve_data\n            t = time\n            values = {}\n          end\n          yield(t, handle_parsed(tag, record, t, values))\n        end\n\n      rescue Fluent::Plugin::Parser::ParserError => e\n        raise e\n      rescue ArgumentError => e\n        raise unless @replace_invalid_sequence\n        raise unless e.message.index(\"invalid byte sequence in\") == 0\n\n        raw_value = raw_value.scrub(REPLACE_CHAR)\n        retry\n      rescue => e\n        raise Fluent::Plugin::Parser::ParserError, \"parse failed #{e.message}\"\n      end\n    end\n\n    def handle_parsed(tag, record, t, values)\n      if values && @inject_key_prefix\n        values = Hash[values.map { |k, v| [@inject_key_prefix + k, v] }]\n      end\n      r = @hash_value_field ? {@hash_value_field => values} : values\n      if @reserve_data\n        r = r ? record.merge(r) : record\n      end\n      r\n    end\n  end\nend\n","sourceCodeStart":91,"sourceCodeEnd":125,"githubUrl":"https://github.com/fluent/fluentd/blob/dd45c6e18dc7be33b5e5a0f0767bf46307ff5626/lib/fluent/plugin/filter_parser.rb#L91-L125","documentation":"Inside filter_one_record, after the configured parser runs: Fluent::Plugin::Parser::ParserError is re-raised unchanged; ArgumentError messages starting with 'invalid byte sequence in' are scrubbed and retried when replace_invalid_sequence is true; every other exception (TypeError from a non-string raw_value, engine-specific errors) is wrapped as ParserError 'parse failed <original message>'. filter_stream's rescue then sends it to emit_error_event when emit_invalid_record_to_error is true (default).","triggerScenarios":"The parser plugin raises something other than a normal mismatch while processing raw_value: key_name holds a non-String value (Array/Hash) fed to a line-based parser, parser-specific runtime errors, or malformed input that is not an 'invalid byte sequence' ArgumentError.","commonSituations":"JSON fields that sometimes contain arrays/objects instead of a string to parse; parsers with format assumptions (apache2, syslog) receiving foreign data; binary/garbage payloads from misconfigured senders.","solutions":["Verify the data actually matches the chosen <parse> type; test a sample record with the same parser settings.","If key_name can hold non-strings, pre-route or convert those events (grep filter or a record_transformer) before the parser filter.","Set replace_invalid_sequence true to survive encoding issues instead of failing.","Keep emit_invalid_record_to_error true and capture @ERROR output to quantify bad records."],"exampleFix":"# before\n<filter app.**>\n  @type parser\n  key_name message\n  <parse>\n    @type apache2\n  </parse>\n</filter>\n# after: parser matches actual payload shape, bad records land in @ERROR\n<filter app.**>\n  @type parser\n  key_name payload\n  emit_invalid_record_to_error true\n  replace_invalid_sequence true\n  <parse>\n    @type json\n  </parse>\n</filter>","handlingStrategy":"try-catch","validationCode":"# smoke-test the parser against representative payloads before deploy\nparser = Fluent::Plugin.new_parser('json')\nparser.configure(parser_config)\nparser.parse(sample_payload) { |t, r| abort 'parser failed' unless r }","typeGuard":"->(v) { v.is_a?(String) } # key_name must hold a string for line-oriented parsers","tryCatchPattern":"begin\n  parser.parse(raw_value) { |t, r| yield(t, r) }\nrescue Fluent::Plugin::Parser::ParserError => e\n  router.emit_error_event(tag, time, record, e) # 'parse failed ...' goes to @ERROR\nend","preventionTips":["Match the <parse> type to the producer's actual format and test with real samples.","Enable replace_invalid_sequence to survive encoding damage.","Route @ERROR to a persistent sink and alert on its growth.","Guard against non-string values in key_name upstream (grep/transform)."],"tags":["fluentd","ruby","filter-parser","parse-error","per-record-error"],"backgroundTag":"parse-format-mismatch","analyzedSha":"dd45c6e18dc7be33b5e5a0f0767bf46307ff5626","analyzedAt":"2026-08-21T16:22:07.332Z","schemaVersion":2},"datasetVersion":"2026-08-21T18:17:14.833Z"}