{"record":{"id":"9582df4bea89844c","repo":"apache/hadoop","slug":"error-closing-producer","errorCode":null,"errorMessage":"Error closing producer","messagePattern":"Error closing producer","errorType":"exception","errorClass":"MetricsException","httpStatus":null,"severity":"warning","filePath":"hadoop-tools/hadoop-kafka/src/main/java/org/apache/hadoop/metrics2/sink/KafkaSink.java","lineNumber":197,"sourceCode":"    } catch (InterruptedException e) {\n      throw new MetricsException(\"Error sending data\", e);\n    } catch (ExecutionException e) {\n      throw new MetricsException(\"Error sending data\", e);\n    }\n  }\n\n  @Override\n  public void flush() {\n    LOG.debug(\"Kafka seems not to have any flush() mechanism!\");\n  }\n\n  @Override\n  public void close() throws IOException {\n    // Close the producer and set it to null.\n    try {\n      producer.close();\n    } catch (RuntimeException e) {\n      throw new MetricsException(\"Error closing producer\", e);\n    } finally {\n      producer = null;\n    }\n  }\n}\n","sourceCodeStart":179,"sourceCodeEnd":203,"githubUrl":"https://github.com/apache/hadoop/blob/2add9630210752f88ceb1bb74eb65e37bf41da8e/hadoop-tools/hadoop-kafka/src/main/java/org/apache/hadoop/metrics2/sink/KafkaSink.java#L179-L203","documentation":"KafkaSink.close() calls producer.close() and wraps any RuntimeException as MetricsException('Error closing producer'), always nulling the producer field in finally. A failing close usually means the producer buffer could not be drained: brokers became unreachable while undelivered records were pending, or the producer was already in a broken state from a prior failure.","triggerScenarios":"Daemon shutdown (metrics system closes sinks) while Kafka is down or slow, so producer.close() times out flushing buffered records and throws InterruptException/TimeoutException (RuntimeExceptions); closing after an earlier send failure left the producer in error state; double-close where the underlying client misbehaves.","commonSituations":"Rolling restarts of Hadoop daemons during a Kafka outage; close races with in-flight putMetrics on another thread; kafka-clients version quirks around close-timeout behavior.","solutions":["Check Kafka health at shutdown time — this error typically accompanies 'Error sending data' entries earlier in the log","Set producer close/linger timeouts appropriately (via additional producer props) so buffered records do not block shutdown","If shutdown noise only, it can be ignored after confirming metrics otherwise flowed; the producer is nulled regardless, preventing leaks"],"exampleFix":null,"handlingStrategy":"try-catch","validationCode":null,"typeGuard":null,"tryCatchPattern":"try {\n  sink.close();\n} catch (IOException | MetricsException e) {\n  // usually shutdown-time flush failure while Kafka is down; producer is nulled\n  // regardless, so no leak — log and continue shutdown\n  LOG.warn(\"Kafka sink close failed\", e);\n}","preventionTips":["Keep Kafka healthy during Hadoop rolling restarts to let producers drain on close","Tune producer linger/close timeouts so shutdown is not blocked by pending batches"],"tags":["hadoop","metrics2","kafka","shutdown","resource-cleanup"],"backgroundTag":"resource-close-failed","analyzedSha":"2add9630210752f88ceb1bb74eb65e37bf41da8e","analyzedAt":"2026-08-22T19:55:07.957Z","schemaVersion":2},"datasetVersion":"2026-08-22T20:17:22.307Z"}