apache/hadoop · warning · MetricsException
Error closing connection to Graphite.
Error message
Error closing connection to Graphite.
What it means
GraphiteSink.flush() calls graphite.flush() every flush cycle. If the flush throws (broken connection), the sink logs WARN "Error flushing metrics to Graphite." and attempts graphite.close(); when the close also throws, MetricsException("Error closing connection to Graphite.", e1) is thrown. As with the putMetrics variant, the surfaced exception is the secondary close failure and the root network error is in the preceding WARN log.
Source
Thrown at hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/metrics2/sink/GraphiteSink.java:119
LOG.warn("Error sending metrics to Graphite.", e);
try {
graphite.close();
} catch (Exception e1) {
throw new MetricsException("Error closing connection to Graphite", e1);
}
}
}
@Override
public void flush() {
try {
graphite.flush();
} catch (Exception e) {
LOG.warn("Error flushing metrics to Graphite.", e);
try {
graphite.close();
} catch (Exception e1) {
throw new MetricsException("Error closing connection to Graphite.", e1);
}
}
}
@Override
public void close() throws IOException {
graphite.close();
}
public static class Graphite {
private final static int MAX_CONNECTION_FAILURES = 5;
private String serverHost;
private int serverPort;
private Writer writer = null;
private Socket socket = null;
private int connectionFailures = 0;
View on GitHub (pinned to 2add963021)
Solutions
- Check the WARN 'Error flushing metrics to Graphite.' line for the underlying exception
- Restore Graphite server availability and verify server_host/server_port connectivity (nc -vz host port)
- Rely on lazy reconnection: the next putMetrics/write cycle re-establishes the socket
- Restart the daemon after remediation if the sink has hit its 5-failure silent-give-up limit
Defensive patterns
Strategy: retry
Validate before calling
try (Socket s = new Socket()) {
s.connect(new InetSocketAddress(graphiteHost, graphitePort), 3000);
} catch (IOException e) {
LOG.warn("Graphite endpoint unhealthy before flush: {}", e.toString());
} Try / catch
try {
sink.flush();
} catch (MetricsException e) {
// flush + close both failed; connection is dead but the next write() reconnects
LOG.warn("Graphite flush/close failed; socket will be re-established on next write", e);
} Prevention
- Schedule Graphite restarts/maintenance with awareness that in-flight sink cycles will log one WARN + one exception each
- Verify endpoint health (nc -vz) after any Graphite migration before assuming the sink recovered
- Watch the failure count: after 5 consecutive failures the sink silently drops metrics until restart
When it happens
Trigger: flush() on a socket already broken by a Graphite restart, connection reset, or idle-timeout firewall drop; the subsequent close() on the dead socket throws as well.
Common situations: Graphite server restarts; LB/NAT idle timeouts shorter than the sink flush interval; network partitions that leave the TCP connection half-open.
Related errors
- Error closing connection to Graphite
- Error creating connection, {}:{}
- Already connected to Graphite
- Error connecting to file system: ${basePath} [${ex}]
- Error writing metric to StatsD
AI-assisted analysis of apache/hadoop@2add963021 (2026-08-22).
Data as JSON: /api/errors/407387e651001ec5.
Report an issue: GitHub.