{"record":{"id":"9f7d8caa0a108f1c","repo":"fluent/fluentd","slug":"can-t-create-buffer-file-for-path-stop-creatin-9f7d8c","errorCode":null,"errorMessage":"can't create buffer file for #{path}. Stop creating buffer files: error = #{e}","messagePattern":"can't create buffer file for #(.+?)\\. Stop creating buffer files: error = #(.+?)","errorType":"exception","errorClass":"BufferOverflowError","httpStatus":null,"severity":"error","filePath":"lib/fluent/plugin/buffer/file_single_chunk.rb","lineNumber":263,"sourceCode":"          k ||= ''\n          URI::RFC2396_PARSER.escape(k, ESCAPE_REGEXP)\n        end\n\n        def decode_key(key)\n          URI::RFC2396_PARSER.unescape(key)\n        end\n\n        def create_new_chunk(path, metadata, perm)\n          @path = self.class.generate_stage_chunk_path(path, encode_key(metadata), @unique_id)\n          begin\n            @chunk = File.open(@path, 'wb+', perm)\n            @chunk.set_encoding(Encoding::ASCII_8BIT)\n            @chunk.sync = true\n            @chunk.binmode\n          rescue => e\n            # Here assumes \"Too many open files\" like recoverable error so raising BufferOverflowError.\n            # If other cases are possible, we will change error handling with proper classes.\n            raise BufferOverflowError, \"can't create buffer file for #{path}. Stop creating buffer files: error = #{e}\"\n          end\n\n          @state = :unstaged\n          @bytesize = 0\n          @commit_position = @chunk.pos # must be 0\n          @adding_bytes = 0\n          @adding_size = 0\n        end\n\n        def load_existing_staged_chunk(path)\n          @path = path\n          raise FileChunkError, \"staged file chunk is empty\" if File.size(@path).zero?\n\n          @chunk = File.open(@path, 'rb+')\n          @chunk.set_encoding(Encoding::ASCII_8BIT)\n          @chunk.sync = true\n          @chunk.binmode\n          @chunk.seek(0, IO::SEEK_END)","sourceCodeStart":245,"sourceCodeEnd":281,"githubUrl":"https://github.com/fluent/fluentd/blob/dd45c6e18dc7be33b5e5a0f0767bf46307ff5626/lib/fluent/plugin/buffer/file_single_chunk.rb#L245-L281","documentation":"When the single-file buffer plugin needs a new staging chunk, FileSingleChunk#create_new_chunk does File.open(stage_path, 'wb+', perm). Any StandardError from that open is re-raised as BufferOverflowError ('can't create buffer file for <path>. Stop creating buffer files: error = <e>') so that fluentd's output retry machinery treats the sink as temporarily overloaded instead of crashing. The code comment states the expected cause is Errno::EMFILE ('Too many open files'); ENOSPC, EACCES, or EROFS on the buffer directory produce the same wrapping.","triggerScenarios":"File.open on the new stage chunk path fails: process fd limit reached (many in_tail instances, many parallel chunks, fd leak), disk full, buffer directory permissions wrong for the fluentd user, or read-only filesystem.","commonSituations":"Default systemd LimitNOFILE=1024 on busy log pipelines; heavy multi-tag routing creating many chunks at once; disk-full on the buffer partition; running fluentd as a user without write access to the buffer dir; SELinux denials.","solutions":["Raise the file descriptor limit for fluentd: systemd [Service] LimitNOFILE=65536 (or ulimit -n in the launcher), then restart.","Verify the buffer directory is writable by the fluentd user and has free space (df -h; sudo -u fluent test -w <dir>).","Reduce simultaneously open chunks: lower queue_limit_length/total_limit_size or flush_thread_count so fewer chunks exist at once.","Inspect lsof -p <fluentd pid> | wc -l for fd leaks coming from other plugins (e.g. very many in_tail file handles)."],"exampleFix":"# before: systemd default limits, EMFILE wrapped as BufferOverflowError\n#   [Service]\n#   LimitNOFILE=1024\n# after:\n#   [Service]\n#   LimitNOFILE=65536","handlingStrategy":"retry","validationCode":"# preflight: check fd headroom and writability before fluentd loads\n`ulimit -n`.to_i.tap { |n| warn \"fd limit low: #{n}\" if n < 65536 }\nrequire 'fileutils'\nFileUtils.touch('/var/log/fluent/buffer/.writetest') rescue warn 'buffer dir not writable'","typeGuard":null,"tryCatchPattern":"begin\n  buffer.emit(title, es)\nrescue Fluent::Plugin::BufferOverflowError => e\n  # transient EMFILE/ENOSPC: fluentd core retries with backoff; alert on repeats\n  log.warn \"buffer file creation failed, will retry: #{e.message}\"\nend","preventionTips":["Set systemd LimitNOFILE=65536 (or equivalent) for fluentd.","Track open fds (lsof -p <pid> | wc -l) and trend it.","Keep buffer disk space monitored; ENOSPC raises the same error.","Bound queue_limit_length/total_limit_size to cap simultaneous open chunks.","Fix fd leaks from other plugins (excess in_tail file handles)."],"tags":["fluentd","ruby","file-descriptors","too-many-open-files","buffer-overflow"],"backgroundTag":"too-many-open-files","analyzedSha":"dd45c6e18dc7be33b5e5a0f0767bf46307ff5626","analyzedAt":"2026-08-21T16:22:07.332Z","schemaVersion":2},"datasetVersion":"2026-08-21T18:17:14.833Z"}