fluent/fluentd · warning · FileChunkError

staged file chunk is empty

Error message

staged file chunk is empty

What it means

During resume, load_existing_staged_chunk handles chunk files in 'staged' state (path matches .b*.): when a companion .meta file exists but the data file size is 0, FileChunkError 'staged file chunk is empty' is raised. A staged chunk with metadata but zero bytes of data is inconsistent (data was never written or was lost), so the chunk cannot be restored.

Source

Thrown at lib/fluent/plugin/buffer/file_chunk.rb:331

            raise BufferOverflowError, "can't create buffer metadata for #{path}. Stop creating buffer files: error = #{e}"
          end

          @state = :unstaged
          @bytesize = 0
          @commit_position = @chunk.pos # must be 0
          @adding_bytes = 0
          @adding_size = 0
        end

        def load_existing_staged_chunk(path)
          @path = path
          @meta_path = @path + '.meta'

          @meta = nil
          # staging buffer chunk without metadata is classic buffer chunk file
          # and it should be enqueued immediately
          if File.exist?(@meta_path)
            raise FileChunkError, "staged file chunk is empty" if File.size(@path).zero?

            @chunk = File.open(@path, 'rb+')
            @chunk.set_encoding(Encoding::ASCII_8BIT)
            @chunk.sync = true
            @chunk.seek(0, IO::SEEK_END)
            @chunk.binmode

            @meta = File.open(@meta_path, 'rb+')
            @meta.set_encoding(Encoding::ASCII_8BIT)
            @meta.sync = true
            @meta.binmode
            begin
              restore_metadata(@meta.read)
            rescue => e
              @chunk.close
              @meta.close
              raise FileChunkError, "staged meta file is broken. #{e.message}"
            end

View on GitHub (pinned to dd45c6e18d)

Solutions

1. Let fluentd handle it: resume catches FileChunkError and routes the file to handle_broken_files (logged and moved/deleted) — the empty chunk is discarded, which loses nothing since the data file was empty
2. Avoid manual truncation/Editing of files inside buffer directories; use rm on whole chunks only if you understand the state
3. Use graceful shutdown (SIGTERM) so staged chunks are flushed/enqueued before exit
4. Verify storage health if zero-length chunks appear repeatedly without crashes

# Pre-start scan: find staged chunks whose data file is empty while a .meta exists
Dir.glob('buffer/**/*.b*.buf').each do |p|
  puts "EMPTY STAGED CHUNK: #{p}" if File.exist?(p + '.meta') && File.size(p).zero?
end
# operator can then archive or remove them before fluentd resumes

begin
  Fluent::Plugin::Buffer::FileChunk.new(metadata, path, :staged)
rescue Fluent::Plugin::Buffer::FileChunkError => e
  # resume() already catches this per file and routes to handle_broken_files (logged + removed)
  log.warn "discarding empty staged chunk #{path}: #{e.message}"
end

When it happens

Trigger: A power failure or SIGKILL after the chunk data file was created and truncated/rolled but before bytes were flushed, while the .meta already existed; file-level corruption that zeroed the data file; manual truncation of chunk files in the buffer directory.

Common situations: Restarting after a hard crash with flush_at_shutdown chunks present; disk issues zeroing files; operators manually 'cleaning' buffer files with > or truncate; buffer dirs on poorly synced network storage.

Related errors

• invalid meta data
• invalid unique_id
• invalid created_at
• invalid modified_at
• can't create buffer file for #{path}. Stop creating buffer f

AI-assisted analysis of fluent/fluentd@dd45c6e18d (2026-08-21). Data as JSON: /api/errors/0b308cdb7d5249d9. Report an issue: GitHub.