{"record":{"id":"0b308cdb7d5249d9","repo":"fluent/fluentd","slug":"staged-file-chunk-is-empty","errorCode":null,"errorMessage":"staged file chunk is empty","messagePattern":"staged file chunk is empty","errorType":"exception","errorClass":"FileChunkError","httpStatus":null,"severity":"warning","filePath":"lib/fluent/plugin/buffer/file_chunk.rb","lineNumber":331,"sourceCode":"            raise BufferOverflowError, \"can't create buffer metadata for #{path}. Stop creating buffer files: error = #{e}\"\n          end\n\n          @state = :unstaged\n          @bytesize = 0\n          @commit_position = @chunk.pos # must be 0\n          @adding_bytes = 0\n          @adding_size = 0\n        end\n\n        def load_existing_staged_chunk(path)\n          @path = path\n          @meta_path = @path + '.meta'\n\n          @meta = nil\n          # staging buffer chunk without metadata is classic buffer chunk file\n          # and it should be enqueued immediately\n          if File.exist?(@meta_path)\n            raise FileChunkError, \"staged file chunk is empty\" if File.size(@path).zero?\n\n            @chunk = File.open(@path, 'rb+')\n            @chunk.set_encoding(Encoding::ASCII_8BIT)\n            @chunk.sync = true\n            @chunk.seek(0, IO::SEEK_END)\n            @chunk.binmode\n\n            @meta = File.open(@meta_path, 'rb+')\n            @meta.set_encoding(Encoding::ASCII_8BIT)\n            @meta.sync = true\n            @meta.binmode\n            begin\n              restore_metadata(@meta.read)\n            rescue => e\n              @chunk.close\n              @meta.close\n              raise FileChunkError, \"staged meta file is broken. #{e.message}\"\n            end","sourceCodeStart":313,"sourceCodeEnd":349,"githubUrl":"https://github.com/fluent/fluentd/blob/dd45c6e18dc7be33b5e5a0f0767bf46307ff5626/lib/fluent/plugin/buffer/file_chunk.rb#L313-L349","documentation":"During resume, load_existing_staged_chunk handles chunk files in 'staged' state (path matches .b*.): when a companion .meta file exists but the data file size is 0, FileChunkError 'staged file chunk is empty' is raised. A staged chunk with metadata but zero bytes of data is inconsistent (data was never written or was lost), so the chunk cannot be restored.","triggerScenarios":"A power failure or SIGKILL after the chunk data file was created and truncated/rolled but before bytes were flushed, while the .meta already existed; file-level corruption that zeroed the data file; manual truncation of chunk files in the buffer directory.","commonSituations":"Restarting after a hard crash with flush_at_shutdown chunks present; disk issues zeroing files; operators manually 'cleaning' buffer files with > or truncate; buffer dirs on poorly synced network storage.","solutions":["Let fluentd handle it: resume catches FileChunkError and routes the file to handle_broken_files (logged and moved/deleted) — the empty chunk is discarded, which loses nothing since the data file was empty","Avoid manual truncation/Editing of files inside buffer directories; use rm on whole chunks only if you understand the state","Use graceful shutdown (SIGTERM) so staged chunks are flushed/enqueued before exit","Verify storage health if zero-length chunks appear repeatedly without crashes"],"exampleFix":null,"handlingStrategy":"validation","validationCode":"# Pre-start scan: find staged chunks whose data file is empty while a .meta exists\nDir.glob('buffer/**/*.b*.buf').each do |p|\n  puts \"EMPTY STAGED CHUNK: #{p}\" if File.exist?(p + '.meta') && File.size(p).zero?\nend\n# operator can then archive or remove them before fluentd resumes","typeGuard":null,"tryCatchPattern":"begin\n  Fluent::Plugin::Buffer::FileChunk.new(metadata, path, :staged)\nrescue Fluent::Plugin::Buffer::FileChunkError => e\n  # resume() already catches this per file and routes to handle_broken_files (logged + removed)\n  log.warn \"discarding empty staged chunk #{path}: #{e.message}\"\nend","preventionTips":["Use SIGTERM/graceful shutdown so staged chunks get data or are enqueued","Never truncate files in the buffer directory manually","Keep the buffer volume healthy (fsck after crashes)","Treat 'broken chunk' log lines at startup as a signal to review shutdown hygiene"],"tags":["fluentd","buffer","file-chunk","resume","empty-chunk","corruption"],"backgroundTag":"corrupt-buffer-metadata","analyzedSha":"dd45c6e18dc7be33b5e5a0f0767bf46307ff5626","analyzedAt":"2026-08-21T16:22:07.332Z","schemaVersion":2},"datasetVersion":"2026-08-21T18:17:14.833Z"}