fluent/fluentd · critical · FileChunkError

enqueued file chunk is empty

Error message

enqueued file chunk is empty

What it means

While restoring the queue at startup, the classic file buffer plugin loads each already-enqueued chunk (q<hex>.buf) via FileChunk#load_existing_enqueued_chunk. An enqueued chunk is expected to contain committed data; if File.size(path) is zero the constructor raises FileChunkError 'enqueued file chunk is empty'. The error stops fluentd from starting because the buffer plugin cannot reconstruct a consistent state.

Source

Thrown at lib/fluent/plugin/buffer/file_chunk.rb:375

          else
            # classic buffer chunk - read only chunk
            @chunk = File.open(@path, 'rb')
            @chunk.set_encoding(Encoding::ASCII_8BIT)
            @chunk.binmode
            @chunk.seek(0, IO::SEEK_SET)
            @state = :queued
            @bytesize = @chunk.size

            restore_metadata_partially(@chunk)

            @commit_position = @chunk.size
            @unique_id = self.class.unique_id_from_path(@path) || @unique_id
          end
        end

        def load_existing_enqueued_chunk(path)
          @path = path
          raise FileChunkError, "enqueued file chunk is empty" if File.size(@path).zero?

          @chunk = File.open(@path, 'rb')
          @chunk.set_encoding(Encoding::ASCII_8BIT)
          @chunk.binmode
          @chunk.seek(0, IO::SEEK_SET)
          @bytesize = @chunk.size
          @commit_position = @chunk.size

          @meta_path = @path + '.meta'
          if File.readable?(@meta_path)
            begin
              restore_metadata(File.open(@meta_path){|f| f.set_encoding(Encoding::ASCII_8BIT); f.binmode; f.read })
            rescue => e
              @chunk.close
              raise FileChunkError, "enqueued meta file is broken. #{e.message}"
            end
          else
            restore_metadata_partially(@chunk)

View on GitHub (pinned to dd45c6e18d)

Solutions

1. Stop fluentd and delete or quarantine the zero-byte q<hex>.buf file (and its sibling .meta if present) named in the startup error, then restart.
2. Investigate why the file was created empty: check disk space, dmesg for OOM/FS errors, and whether another process wrote into the buffer dir.
3. If it recurs, check filesystem health (fsck) and move the buffer to a dedicated, monitored partition.

Example fix

# before: fluentd fails to start on 'enqueued file chunk is empty'
# after: remove the empty queued chunk and restart
sudo systemctl stop fluentd
find /var/log/fluent/buffer -name 'q*.buf' -size 0 -print -delete
sudo systemctl start fluentd

# preflight: flag zero-size queued chunk files before startup
BUFFER_DIR = '/var/log/fluent/buffer'
Dir.glob("#{BUFFER_DIR}/*").each do |f|
  next if f.end_with?('.meta')
  warn "empty chunk file: #{f}" if File.size(f).zero?
end

begin
  chunk = Fluent::Plugin::Buffer::FileChunk.new(metadata, path, :queued)
rescue Fluent::Plugin::Buffer::FileChunkError => e
  log.error "removing empty queued chunk #{path}: #{e.message}"
  File.delete(path) if File.size(path).zero?
end

When it happens

Trigger: A q<hex>.buf file of 0 bytes exists in the buffer directory: the chunk was renamed to queued state but its data write never completed (crash between enqueue and flush), a disk-full short write, manual truncation, or a half-copied buffer directory.

Common situations: Hard crashes or OOM kills during enqueue/flush; buffer filesystem that ran out of space or went read-only; rsync/scp of a live buffer directory; leftover files after experiments with buffer path settings.

Related errors

• staged file chunk is empty
• enqueued file chunk is empty
• staged meta file is broken. #{e.message}
• enqueued meta file is broken. #{e.message}
• Invalid chunk found. unique_id and key not exist: #{@path}

AI-assisted analysis of fluent/fluentd@dd45c6e18d (2026-08-21). Data as JSON: /api/errors/342fd077993ca0f1. Report an issue: GitHub.