{"record":{"id":"9f847b7098bd43a0","repo":"fluent/fluentd","slug":"can-t-create-buffer-file-for-path-stop-creatin","errorCode":null,"errorMessage":"can't create buffer file for #{path}. Stop creating buffer files: error = #{e}","messagePattern":"can't create buffer file for #(.+?)\\. Stop creating buffer files: error = #(.+?)","errorType":"exception","errorClass":"BufferOverflowError","httpStatus":null,"severity":"critical","filePath":"lib/fluent/plugin/buffer/file_chunk.rb","lineNumber":293,"sourceCode":"          file.set_encoding(Encoding::ASCII_8BIT)\n          file.sync = true\n          file.binmode\n          file.pos = pos\n          callback.call(file) if callback\n        end\n\n        def create_new_chunk(path, perm)\n          @path = self.class.generate_stage_chunk_path(path, @unique_id)\n          @meta_path = @path + '.meta'\n          begin\n            @chunk = File.open(@path, 'wb+', perm)\n            @chunk.set_encoding(Encoding::ASCII_8BIT)\n            @chunk.sync = true\n            @chunk.binmode\n          rescue => e\n            # Here assumes \"Too many open files\" like recoverable error so raising BufferOverflowError.\n            # If other cases are possible, we will change error handling with proper classes.\n            raise BufferOverflowError, \"can't create buffer file for #{path}. Stop creating buffer files: error = #{e}\"\n          end\n          begin\n            @meta = File.open(@meta_path, 'wb', perm)\n            @meta.set_encoding(Encoding::ASCII_8BIT)\n            @meta.sync = true\n            @meta.binmode\n            write_metadata(update: false)\n          rescue => e\n            # This case is easier than enqueued!. Just removing pre-create buffer file\n            @chunk.close rescue nil\n            File.unlink(@path) rescue nil\n\n            if @meta\n              # ensure to unlink when #write_metadata fails\n              @meta.close rescue nil\n              File.unlink(@meta_path) rescue nil\n            end\n","sourceCodeStart":275,"sourceCodeEnd":311,"githubUrl":"https://github.com/fluent/fluentd/blob/dd45c6e18dc7be33b5e5a0f0767bf46307ff5626/lib/fluent/plugin/buffer/file_chunk.rb#L275-L311","documentation":"FileChunk#create_new_chunk opens a new staged chunk data file with File.open(path,'wb+',perm); any raised StandardError is converted to BufferOverflowError with this message. The code comment states the assumption is a recoverable 'Too many open files' (EMFILE) condition — raising BufferOverflowError lets the output's overflow_action machinery (retry/block/drop) deal with it rather than crashing the agent.","triggerScenarios":"Process file-descriptor exhaustion (ulimit -n reached) when creating buffer chunk files — many buffers, many flush threads, or an FD leak; also any IO error opening the file such as ENOSPC (disk full), EACCES (permissions on the buffer dir), or ENAMETOOLONG.","commonSituations":"High-throughput fluentd with several file/file_single buffers hitting the default 1024 FD limit; buffer volume filled (ENOSPC); dir_permission/file_permission mismatch after a chmod; long-running FD leak in a custom plugin.","solutions":["Check the trailing 'error = ...' in the message: Errno::EMFILE → raise the fd limit (ulimit -n / LimitNOFILE) ; Errno::ENOSPC → free disk or lower total_limit_size; Errno::EACCES → fix dir ownership/permissions","Reduce concurrent open files: fewer flush threads, fewer separate buffer paths, larger chunk_limit_size so fewer chunks exist","Verify with ls /proc/$(pidof fluentd)/fd | wc -l against ulimit -n","If it is a genuine FD leak in a plugin, capture lsof output over time and fix the leak"],"exampleFix":"# systemd: raise fd limit for fluentd\n# /etc/systemd/system/fluentd.service\n[Service]\nLimitNOFILE=65536\n\n# shell check before/while running\nulimit -n\nls /proc/$(pgrep -f 'fluentd.*ruby')/fd | wc -l","handlingStrategy":"retry","validationCode":"# Pre-start: verify fd headroom and writable buffer dir\nlimits = Process.getrlimit(:NOFILE)\nabort \"NOFILE too low: #{limits[0]}\" if limits[0] < 65_536\nbuf_dir = File.dirname(buffer_path)\ntest = File.join(buf_dir, \".fd_probe_#{Process.pid}\")\nFile.open(test,'wb'){|f| f.write('x')} ; File.unlink(test)\n# runtime: watch fd usage\nfds = Dir.glob(\"/proc/#{Process.pid}/fd\").size\nabort \"fd usage #{fds} near limit #{limits[0]}\" if fds > limits[0] * 0.8","typeGuard":null,"tryCatchPattern":"begin\n  buffer.write(metadata => data)\nrescue Fluent::Plugin::Buffer::BufferOverflowError => e\n  if e.message.include?(\"can't create buffer file\")\n    sleep 1 and retry   # EMFILE/ENOSPC class: transient; fix ulimit/disk while retrying\n  else\n    raise               # genuine buffer-full: apply overflow_action\n  end\nend","preventionTips":["Set LimitNOFILE=65536 (systemd) or ulimit -n for the fluentd process","Keep total_limit_size well under the buffer volume's free space","Watch open-FD count and buffer-disk usage with alerts","Fewer, larger chunks (bigger chunk_limit_size) means fewer simultaneously open files"],"tags":["fluentd","buffer","file-chunk","fd-limit","emfile","disk-full","buffer-overflow-error"],"backgroundTag":"too-many-open-files","analyzedSha":"dd45c6e18dc7be33b5e5a0f0767bf46307ff5626","analyzedAt":"2026-08-21T16:22:07.332Z","schemaVersion":2},"datasetVersion":"2026-08-21T18:17:14.833Z"}