fluent/fluentd · critical · BufferOverflowError

can't create buffer metadata for #{path}. Stop creating buff

Error message

can't create buffer metadata for #{path}. Stop creating buffer files: error = #{e}

What it means

In create_new_chunk, after the data file is opened successfully, the companion .meta file is opened and initial metadata written; any failure there triggers cleanup (chunk file closed and unlinked, meta unlinked) and re-raise as BufferOverflowError with this message. As with the data-file case, the intended trigger is FD exhaustion, but any meta write error (disk full, permission) surfaces here too. Cleanup means no orphan half-chunk remains.

Source

Thrown at lib/fluent/plugin/buffer/file_chunk.rb:313

          begin
            @meta = File.open(@meta_path, 'wb', perm)
            @meta.set_encoding(Encoding::ASCII_8BIT)
            @meta.sync = true
            @meta.binmode
            write_metadata(update: false)
          rescue => e
            # This case is easier than enqueued!. Just removing pre-create buffer file
            @chunk.close rescue nil
            File.unlink(@path) rescue nil

            if @meta
              # ensure to unlink when #write_metadata fails
              @meta.close rescue nil
              File.unlink(@meta_path) rescue nil
            end

            # Same as @chunk case. See above
            raise BufferOverflowError, "can't create buffer metadata for #{path}. Stop creating buffer files: error = #{e}"
          end

          @state = :unstaged
          @bytesize = 0
          @commit_position = @chunk.pos # must be 0
          @adding_bytes = 0
          @adding_size = 0
        end

        def load_existing_staged_chunk(path)
          @path = path
          @meta_path = @path + '.meta'

          @meta = nil
          # staging buffer chunk without metadata is classic buffer chunk file
          # and it should be enqueued immediately
          if File.exist?(@meta_path)
            raise FileChunkError, "staged file chunk is empty" if File.size(@path).zero?

View on GitHub (pinned to dd45c6e18d)

Solutions

1. Read the wrapped 'error = Errno::...' to classify: EMFILE → raise ulimit/LimitNOFILE and reduce concurrent chunks; ENOSPC → free space, cap total_limit_size, move buffer to a bigger volume; EACCES/EROFS → fix mount/permissions on the buffer directory
2. Because cleanup unlinks the half-created files, no manual file removal is needed — after fixing the cause, buffered output recovers on its own
3. Monitor buffer volume free space and open-FD count with alerts
4. Consider memory buffer for small, non-critical outputs to reduce file pressure

Example fix

# /etc/security/limits.conf or systemd unit
[Service]
LimitNOFILE=65536

# fluent.conf: keep buffer within real disk capacity
<buffer>
  @type file
  total_limit_size 4GB   # sized below the volume's free space
</buffer>

free = Sys::Filesystem.stat(buffer_dir).bytes_free
abort 'buffer disk nearly full' if free < total_limit_size
Process.getrlimit(:NOFILE).tap { |(soft,)| abort 'raise NOFILE' if soft < 65_536 }

begin
  buffer.write(metadata => data)
rescue Fluent::Plugin::Buffer::BufferOverflowError => e
  if e.message.include?("can't create buffer metadata")
    log.warn 'meta creation failed (fd/disk/perm); retrying after backoff'
    sleep 2 and retry
  else
    raise
  end
end

When it happens

Trigger: File.open on path+'.meta' or write_metadata fails: EMFILE when FDs are exhausted, ENOSPC when the disk fills between creating the chunk file and its meta, EACCES/EROFS on read-only or mis-permitted buffer directories.

Common situations: Disk filling rapidly under heavy buffering (meta writes are small but frequent); FD pressure from many staged chunks; container mounts that became read-only; dir_permission config not matching actual directory mode.

Related errors

• can't create buffer file for #{path}. Stop creating buffer f
• invalid meta data
• invalid unique_id
• invalid created_at
• invalid modified_at

AI-assisted analysis of fluent/fluentd@dd45c6e18d (2026-08-21). Data as JSON: /api/errors/baafe6ae2ea20e57. Report an issue: GitHub.