fluent/fluentd · error · NotImplementedError

Optional API #parse_partial_data is not implemented

Error message

Optional API #parse_partial_data is not implemented

What it means

Second optional parser API guard (lib/fluent/plugin/parser.rb:178): base Parser#parse_partial_data raises NotImplementedError unless the concrete class overrides it (msgpack does via alias). It exists for parsers that can consume partial bytes from a stream; in_exec/out_exec_filter call it behind implement?(:parse_partial_data). Direct invocation on an unsupported parser triggers this error.

Source

Thrown at lib/fluent/plugin/parser.rb:178

        parse(*a, &b)
      end

      def implement?(feature)
        methods_of_plugin = self.class.instance_methods(false)
        case feature
        when :parse_io then methods_of_plugin.include?(:parse_io)
        when :parse_partial_data then methods_of_plugin.include?(:parse_partial_data)
        else
          raise ArgumentError, "Unknown feature for parser plugin: #{feature}"
        end
      end

      def parse_io(io, &block)
        raise NotImplementedError, "Optional API #parse_io is not implemented"
      end

      def parse_partial_data(data, &block)
        raise NotImplementedError, "Optional API #parse_partial_data is not implemented"
      end

      def parse_time(record)
        if @time_key && record.respond_to?(:has_key?) && record.has_key?(@time_key)
          src = if @keep_time_key
                  record[@time_key]
                else
                  record.delete(@time_key)
                end
          @time_parser.parse(src)
        elsif @estimate_current_event
          Fluent::EventTime.now
        else
          nil
        end
      rescue Fluent::TimeParser::TimeParseError => e
        raise ParserError, e.message
      end

View on GitHub (pinned to dd45c6e18d)

Solutions

1. Guard every call with parser.implement?(:parse_partial_data) and fall back to parser.parse(data).
2. For binary/streaming sources prefer @type msgpack, which implements parse_partial_data.
3. In custom parsers, define def parse_partial_data(data, &block) rather than relying on the inherited base.

Example fix

# before
@parser.parse_partial_data(io.readpartial(@read_block_size), &method(:on_record))
# after
if @parser.implement?(:parse_partial_data)
  @parser.parse_partial_data(io.readpartial(@read_block_size), &method(:on_record))
else
  @parser.parse(io.readpartial(@read_block_size), &method(:on_record))
end

if @parser.implement?(:parse_partial_data)
  @parser.parse_partial_data(data, &callback)
else
  @parser.parse(data, &callback)
end

def partial_data_capable?(parser)
  parser.class.instance_methods(false).include?(:parse_partial_data)
end

begin
  @parser.parse_partial_data(chunk, &callback)
rescue NotImplementedError
  @parser.parse(chunk, &callback)
end

When it happens

Trigger: Calling parser.parse_partial_data(chunk) on @type json/regexp/csv parsers; custom dispatch code that skips the implement?(:parse_partial_data) check; a subclass defining parse_partial_data= as an assignment or with different arity so the base method remains callable.

Common situations: Custom plugins imitating out_exec_filter's readpartial loop but calling parse_partial_data unconditionally; fluentd version changes where a parser type dropped/gained the method; monkey-patches registering parsers through compat layers that hide real method definitions.

Related errors

• Optional API #parse_io is not implemented
• Implement this method in child class
• Unknown feature for parser plugin: #{feature}
• unknown value conversion for key:'#{field_name}', type:'#{ty
• Implement this method in child class

AI-assisted analysis of fluent/fluentd@dd45c6e18d (2026-08-21). Data as JSON: /api/errors/4889ac5a94c17a72. Report an issue: GitHub.