fluent/fluentd · error · FileChunkError

invalid created_at

Error message

invalid created_at

What it means

restore_metadata requires the :c (created_at) field to be a positive integer timestamp; data[:c].to_i > 0 must hold or FileChunkError 'invalid created_at' is raised. A metadata Hash that parses but carries a zero/missing/negative created_at is considered broken.

Source

Thrown at lib/fluent/plugin/buffer/file_chunk.rb:224

        # used only for queued v0.12 buffer path or broken files
        def self.unique_id_from_path(path)
          if /\.(b|q)([0-9a-f]+)\.[^\/]*\Z/n =~ path # //n switch means explicit 'ASCII-8BIT' pattern
            return $2.scan(/../).map{|x| x.to_i(16) }.pack('C*')
          end
          nil
        end

        def restore_metadata(bindata)
          data = restore_metadata_with_new_format(bindata)

          unless data
            # old type of restore
            data = Fluent::MessagePackFactory.msgpack_unpacker(symbolize_keys: true).feed(bindata).read rescue {}
          end
          raise FileChunkError, "invalid meta data" if data.nil? || !data.is_a?(Hash)
          raise FileChunkError, "invalid unique_id" unless data[:id]
          raise FileChunkError, "invalid created_at" unless data[:c].to_i > 0
          raise FileChunkError, "invalid modified_at" unless data[:m].to_i > 0

          now = Fluent::Clock.real_now

          @unique_id = data[:id]
          @size = data[:s] || 0
          @created_at = data[:c]
          @modified_at = data[:m]

          @metadata.timekey = data[:timekey]
          @metadata.tag = data[:tag]
          @metadata.variables = data[:variables]
          @metadata.seq = data[:seq] || 0
        end

        def restore_metadata_partially(chunk)
          @unique_id = self.class.unique_id_from_path(chunk.path) || @unique_id
          @size = 0

View on GitHub (pinned to dd45c6e18d)

Solutions

1. Let the resume path quarantine the chunk (handle_broken_files) and note the possible data loss in logs
2. Restore from backup if needed
3. Harden the environment: graceful restarts, disk space alerts, avoiding shared buffer dirs across versions
4. Verify system clock sanity (NTP) — wild timestamps in tooling that rewrote metas can also produce this

Dir.glob('buffer/**/*.meta').each do |m|
  data = Fluent::MessagePackFactory.msgpack_unpacker(symbolize_keys: true)
                .feed(File.binread(m)).read rescue {}
  warn "#{m}: invalid created_at" unless data.is_a?(Hash) && data[:c].to_i > 0
end

begin
  Fluent::Plugin::Buffer::FileChunk.new(metadata, path, :staged)
rescue Fluent::Plugin::Buffer::FileChunkError => e
  log.error "quarantining #{path}: #{e.message}"; File.rename(path, "#{path}.broken")
end

When it happens

Trigger: A restored Hash from the old-format fallback where :c is absent ({} gives nil.to_i == 0) or was serialized as 0/garbage; meta files touched by external tooling that rewrote timestamps.

Common situations: Same corruption class as the other metadata validation failures: power failure, disk full, partial writes, version-mismatched buffer directories.

Related errors

• invalid modified_at
• invalid meta data
• invalid unique_id
• staged file chunk is empty
• can't create buffer file for #{path}. Stop creating buffer f

AI-assisted analysis of fluent/fluentd@dd45c6e18d (2026-08-21). Data as JSON: /api/errors/8aa90c14c97d7cb1. Report an issue: GitHub.