{"record":{"id":"9b6e16cd381165a3","repo":"fluent/fluentd","slug":"failed-to-flush-the-buffer","errorCode":null,"errorMessage":"failed to flush the buffer.","messagePattern":"failed to flush the buffer\\.","errorType":"console","errorClass":null,"httpStatus":null,"severity":"error","filePath":"lib/fluent/plugin/output.rb","lineNumber":1389,"sourceCode":"            @retry.step\n          else\n            @retry.recalc_next_time # to prevent all flush threads from retrying at the same time\n          end\n\n          if @retry.limit?\n            handle_limit_reached(error)\n          elsif error\n            log_retry_error(error, chunk_id_hex, using_secondary)\n          end\n        end\n      end\n\n      def log_retry_error(error, chunk_id_hex, using_secondary)\n        return unless error\n        if using_secondary\n          msg = \"failed to flush the buffer with secondary output.\"\n        else\n          msg = \"failed to flush the buffer.\"\n        end\n        log.warn(msg, retry_times: @retry.steps, next_retry_time: @retry.next_time.round, chunk: chunk_id_hex, error: error)\n        log.warn_backtrace(error.backtrace)\n      end\n\n      def handle_limit_reached(error)\n        if error\n          records = @buffer.queued_records\n          msg = \"Hit limit for retries. dropping all chunks in the buffer queue.\"\n          log.error msg, retry_times: @retry.steps, records: records, error: error\n          log.error_backtrace error.backtrace\n        end\n        @buffer.clear_queue!\n        log.debug \"buffer queue cleared\"\n        @retry = nil\n      end\n\n      def retry_state(randomize)","sourceCodeStart":1371,"sourceCodeEnd":1407,"githubUrl":"https://github.com/fluent/fluentd/blob/dd45c6e18dc7be33b5e5a0f0767bf46307ff5626/lib/fluent/plugin/output.rb#L1371-L1407","documentation":"Emitted by Fluent::Plugin::Output#log_retry_error every time a buffered chunk fails to flush to the primary output (or to the secondary when using_secondary is true) and is queued for another retry attempt. The same log line carries the underlying exception in the `error:` field plus retry_times and next_retry_time, so the root cause is the wrapped exception, not this message. Data is not lost at this point; it is only dropped later if retries are exhausted ('Hit limit for retries. dropping all chunks in the buffer queue.').","triggerScenarios":"Any output plugin's write/try_write raising while a buffered chunk flushes: out_forward to an unreachable/refusing node, out_http getting non-2xx on a retryable code, TLS handshake failure, DNS resolution failure, destination disk full, or authentication failure against the destination.","commonSituations":"Destination Fluentd/HTTP server temporarily down during deploys, wrong port or TLS mismatch between forward peers, network partitions, full disk on the receiver, or retry_timeout/buffer queue limits sized too small for the outage duration.","solutions":["Read the `error:` field and backtrace on the same log line and fix that root cause first (connectivity, TLS, credentials, destination capacity).","Verify the destination is reachable and healthy from the Fluentd host (telnet/curl to the configured host:port).","Configure a <secondary> output (e.g. local file or alternate forward target) so chunks survive destination outages instead of being dropped after retry limits.","Tune retry parameters (retry_timeout, retry_max_times, retry_secondary_threshold) and buffer limits (queued_chunks_limit_size, total_limit_size) to ride out expected outage windows.","Confirm the buffer path is writable and has free space if using file buffer."],"exampleFix":"# before\n<match **>\n  @type forward\n  <server>\n    host downstream.example.com\n    port 24224\n  </server>\n  <buffer>\n    @type file\n    path /var/log/fluent/buffer\n  </buffer>\n</match>\n\n# after (add retry resilience + secondary so flush failures do not end in dropped chunks)\n<match **>\n  @type forward\n  <server>\n    host downstream.example.com\n    port 24224\n  </server>\n  <buffer>\n    @type file\n    path /var/log/fluent/buffer\n    retry_timeout 72h\n    retry_max_times 30\n  </buffer>\n  <secondary>\n    @type file\n    path /var/log/fluent/fallback\n  </secondary>\n</match>","handlingStrategy":"retry","validationCode":"# Pre-flight check that the forward destination accepts connections before deploying\nrequire 'socket'\nbegin\n  sock = TCPSocket.new('downstream.example.com', 24224)\n  sock.close\n  puts 'destination reachable'\nrescue Errno::ECONNREFUSED, Errno::EHOSTUNREACH, SocketError => e\n  puts \"destination down: #{e.class} #{e.message}\"\nend","typeGuard":null,"tryCatchPattern":null,"preventionTips":["Configure a <secondary> output so flush failures never end in dropped chunks.","Size retry_timeout and buffer total_limit_size to outlive your longest expected destination outage.","Monitor for the follow-up 'Hit limit for retries' error message — that is the actual data-loss signal.","Run destination reachability checks (host/port/TLS) in health monitoring, not just at deploy time."],"tags":["buffering","retry","output-plugin","data-loss-risk"],"backgroundTag":"buffer-flush-retry-failed","analyzedSha":"dd45c6e18dc7be33b5e5a0f0767bf46307ff5626","analyzedAt":"2026-08-21T16:22:07.332Z","schemaVersion":2},"datasetVersion":"2026-08-21T18:17:14.833Z"}