fluent/fluentd · error · BufferOverflowError

can't create buffer file for #{path}. Stop creating buffer f

Error message

can't create buffer file for #{path}. Stop creating buffer files: error = #{e}

What it means

When the single-file buffer plugin needs a new staging chunk, FileSingleChunk#create_new_chunk does File.open(stage_path, 'wb+', perm). Any StandardError from that open is re-raised as BufferOverflowError ('can't create buffer file for <path>. Stop creating buffer files: error = <e>') so that fluentd's output retry machinery treats the sink as temporarily overloaded instead of crashing. The code comment states the expected cause is Errno::EMFILE ('Too many open files'); ENOSPC, EACCES, or EROFS on the buffer directory produce the same wrapping.

Source

Thrown at lib/fluent/plugin/buffer/file_single_chunk.rb:263

          k ||= ''
          URI::RFC2396_PARSER.escape(k, ESCAPE_REGEXP)
        end

        def decode_key(key)
          URI::RFC2396_PARSER.unescape(key)
        end

        def create_new_chunk(path, metadata, perm)
          @path = self.class.generate_stage_chunk_path(path, encode_key(metadata), @unique_id)
          begin
            @chunk = File.open(@path, 'wb+', perm)
            @chunk.set_encoding(Encoding::ASCII_8BIT)
            @chunk.sync = true
            @chunk.binmode
          rescue => e
            # Here assumes "Too many open files" like recoverable error so raising BufferOverflowError.
            # If other cases are possible, we will change error handling with proper classes.
            raise BufferOverflowError, "can't create buffer file for #{path}. Stop creating buffer files: error = #{e}"
          end

          @state = :unstaged
          @bytesize = 0
          @commit_position = @chunk.pos # must be 0
          @adding_bytes = 0
          @adding_size = 0
        end

        def load_existing_staged_chunk(path)
          @path = path
          raise FileChunkError, "staged file chunk is empty" if File.size(@path).zero?

          @chunk = File.open(@path, 'rb+')
          @chunk.set_encoding(Encoding::ASCII_8BIT)
          @chunk.sync = true
          @chunk.binmode
          @chunk.seek(0, IO::SEEK_END)

View on GitHub (pinned to dd45c6e18d)

Solutions

1. Raise the file descriptor limit for fluentd: systemd [Service] LimitNOFILE=65536 (or ulimit -n in the launcher), then restart.
2. Verify the buffer directory is writable by the fluentd user and has free space (df -h; sudo -u fluent test -w <dir>).
3. Reduce simultaneously open chunks: lower queue_limit_length/total_limit_size or flush_thread_count so fewer chunks exist at once.
4. Inspect lsof -p <fluentd pid> | wc -l for fd leaks coming from other plugins (e.g. very many in_tail file handles).

Example fix

# before: systemd default limits, EMFILE wrapped as BufferOverflowError
#   [Service]
#   LimitNOFILE=1024
# after:
#   [Service]
#   LimitNOFILE=65536

# preflight: check fd headroom and writability before fluentd loads
`ulimit -n`.to_i.tap { |n| warn "fd limit low: #{n}" if n < 65536 }
require 'fileutils'
FileUtils.touch('/var/log/fluent/buffer/.writetest') rescue warn 'buffer dir not writable'

begin
  buffer.emit(title, es)
rescue Fluent::Plugin::BufferOverflowError => e
  # transient EMFILE/ENOSPC: fluentd core retries with backoff; alert on repeats
  log.warn "buffer file creation failed, will retry: #{e.message}"
end

When it happens

Trigger: File.open on the new stage chunk path fails: process fd limit reached (many in_tail instances, many parallel chunks, fd leak), disk full, buffer directory permissions wrong for the fluentd user, or read-only filesystem.

Common situations: Default systemd LimitNOFILE=1024 on busy log pipelines; heavy multi-tag routing creating many chunks at once; disk-full on the buffer partition; running fluentd as a user without write access to the buffer dir; SELinux denials.

Related errors

• buffer space has too many data
• staged meta file is broken. #{e.message}
• enqueued file chunk is empty
• enqueued meta file is broken. #{e.message}
• Invalid chunk found. unique_id and key not exist: #{@path}

AI-assisted analysis of fluent/fluentd@dd45c6e18d (2026-08-21). Data as JSON: /api/errors/9f7d8caa0a108f1c. Report an issue: GitHub.