fluent/fluentd · error · Fluent::ConfigError

Invalid contents (not object) in plugin storage file: '#

Error message

Invalid contents (not object) in plugin storage file: '#{@path}'

What it means

LocalStorage#configure parses the existing storage file as JSON and requires the top-level value to be a Hash (storage_local.rb:93-94); a JSON array, string, number, or true/false triggers this Fluent::ConfigError. Note a subtlety: because Fluent::ConfigError inherits from StandardError and this raise sits inside the begin/rescue at lines 87-98, the outer rescue => e catches it and re-raises as "Unexpected error: failed to read data..." (error 248), so the message you see at top level is often the 248 one while this is the root cause logged via log.error.

Solutions

  1. Rewrite the storage file as a JSON object of key/value pairs, e.g. {"key":"value"}, preserving the keys the plugin expects
  2. If the saved state is disposable, stop fluentd, move/delete the offending file, and let LocalStorage recreate it (state resets to empty)
  3. Check the fluentd log for the preceding 'failed to read data from plugin storage file' entry with the original parse error to confirm the file content
  4. Validate the file offline before restart: ruby -rjson -e 'puts JSON.parse(File.read(ARGV[0])).class' storage.json should print Hash

Example fix

# before
$ cat /var/log/fluent/storage.json
["saved_keys"]
# => Invalid contents (not object) in plugin storage file

# after
$ cat /var/log/fluent/storage.json
{"last_save_time":"2024-01-01 00:00:00 UTC"}
Defensive patterns

Strategy: validation

Validate before calling

require 'json'
path = '/var/log/fluent/storage.json'
if File.exist?(path) && !File.empty?(path)
  data = JSON.parse(File.read(path))
  raise "storage file must hold a JSON object, got #{data.class}" unless data.is_a?(Hash)
end

Type guard

def valid_storage_file?(path)
  return true unless File.exist?(path) && !File.empty?(path)
  JSON.parse(File.read(path)).is_a?(Hash)
rescue JSON::ParserError
  false
end

Try / catch

begin
  plugin.configure(conf)
rescue Fluent::ConfigError => e
  if e.message.include?('failed to read data from plugin storage file')
    # root cause logged just above; inspect/repair or remove the JSON file
    exit 1
  end
  raise
end

Prevention

When it happens

Trigger: A storage file containing e.g. [1,2,3], "text", 42, or null — valid JSON but not an object. Happens when a JSON array state file was written by another tool, a manual edit replaced the object with a scalar, or the wrong file was pointed at via path.

Common situations: Operators hand-editing storage.json and replacing the braces with an array; scripts that regenerate state files and emit the wrong top-level type; pointing path at an unrelated JSON document; truncation during a crash followed by partial repair that yielded non-object JSON.

Related errors

AI-assisted analysis of fluent/fluentd@dd45c6e18d (2026-08-21). Data as JSON: /api/errors/d1b0abf44e8e3ee6. Report an issue: GitHub.

Appendix: source

Thrown at lib/fluent/plugin/storage_local.rb:94

            end
            @on_memory = true
            @multi_workers_available = true
          end
        end

        if !@on_memory
          dir = File.dirname(@path)
          FileUtils.mkdir_p(dir, mode: @dir_mode) unless Dir.exist?(dir)
          if File.exist?(@path)
            raise Fluent::ConfigError, "Plugin storage path '#{@path}' is not readable/writable" unless File.readable?(@path) && File.writable?(@path)
            begin
              data = File.open(@path, 'r:utf-8:utf-8') { |io| io.read }
              if data.empty?
                log.warn "detect empty plugin storage file during startup. Ignored: #{@path}"
                return
              end
              data = JSON.parse(data, Fluent::DEFAULT_JSON_PARSE_OPTIONS)
              raise Fluent::ConfigError, "Invalid contents (not object) in plugin storage file: '#{@path}'" unless data.is_a?(Hash)
            rescue => e
              log.error "failed to read data from plugin storage file", path: @path, error: e
              raise Fluent::ConfigError, "Unexpected error: failed to read data from plugin storage file: '#{@path}'"
            end
          else
            raise Fluent::ConfigError, "Directory is not writable for plugin storage file '#{@path}'" unless File.stat(dir).writable?
          end
        end
      end

      def multi_workers_ready?
        unless @multi_workers_available
          log.error "local plugin storage with multi workers should be configured to use directory 'path', or system root_dir and plugin id"
        end
        @multi_workers_available
      end

      def load

View on GitHub (pinned to dd45c6e18d)