fluent/fluentd · critical · FileChunkError

staged file chunk is empty

Error message

staged file chunk is empty

What it means

For the single-file buffer plugin (buf_file_single), FileSingleChunk#load_existing_staged_chunk re-opens staged chunk files (prefix.<key>.b<hex>.<suffix>) at startup. A staged chunk must already contain appended data; if File.size(path) is zero it raises FileChunkError 'staged file chunk is empty', which prevents fluentd from starting.

Source

Thrown at lib/fluent/plugin/buffer/file_single_chunk.rb:275

            @chunk.set_encoding(Encoding::ASCII_8BIT)
            @chunk.sync = true
            @chunk.binmode
          rescue => e
            # Here assumes "Too many open files" like recoverable error so raising BufferOverflowError.
            # If other cases are possible, we will change error handling with proper classes.
            raise BufferOverflowError, "can't create buffer file for #{path}. Stop creating buffer files: error = #{e}"
          end

          @state = :unstaged
          @bytesize = 0
          @commit_position = @chunk.pos # must be 0
          @adding_bytes = 0
          @adding_size = 0
        end

        def load_existing_staged_chunk(path)
          @path = path
          raise FileChunkError, "staged file chunk is empty" if File.size(@path).zero?

          @chunk = File.open(@path, 'rb+')
          @chunk.set_encoding(Encoding::ASCII_8BIT)
          @chunk.sync = true
          @chunk.binmode
          @chunk.seek(0, IO::SEEK_END)

          restore_metadata

          @state = :staged
          @bytesize = @chunk.size
          @commit_position = @chunk.pos
          @adding_bytes = 0
          @adding_size = 0
        end

        def load_existing_enqueued_chunk(path)
          @path = path

View on GitHub (pinned to dd45c6e18d)

Solutions

1. Stop fluentd and delete or quarantine the zero-byte staged file named in the error (and any related .meta), then restart.
2. Check why it was created empty: disk space, OOM kills in dmesg, crash timing during first append.
3. Keep the buffer on a healthy, monitored partition and use graceful restarts (SIGTERM with flush) to avoid partial writes.

Example fix

# before: startup fails on 'staged file chunk is empty'
# after: remove the empty staged chunk and restart
sudo systemctl stop fluentd
find /var/log/fluent/buffer -name '*.b*.buf' -size 0 -print -delete
sudo systemctl start fluentd

# preflight: zero-size staged chunk files (single-mode buffer)
Dir.glob('/var/log/fluent/buffer/*.b*.buf').each do |f|
  warn "empty staged chunk: #{f}" if File.size(f).zero?
end

begin
  chunk = Fluent::Plugin::Buffer::FileSingleChunk.new(metadata, path, :staged, key)
rescue Fluent::Plugin::Buffer::FileChunkError => e
  log.error "removing empty staged chunk #{path}: #{e.message}"
  File.delete(path) if File.size(path).zero?
end

When it happens

Trigger: A staged chunk file of 0 bytes in the buffer directory: fluentd crashed between creating the stage file and the first write/flush, a disk-full short write occurred, or the file was truncated/copied incompletely.

Common situations: kill -9/OOM during high-throughput ingestion; buffer partition full or read-only; partial rsync of the buffer dir; test artifacts left in the buffer path.

Related errors

• enqueued file chunk is empty
• enqueued file chunk is empty
• staged meta file is broken. #{e.message}
• enqueued meta file is broken. #{e.message}
• Invalid chunk found. unique_id and key not exist: #{@path}

AI-assisted analysis of fluent/fluentd@dd45c6e18d (2026-08-21). Data as JSON: /api/errors/e1313630c1a22b13. Report an issue: GitHub.