apache/hadoop · warning · MetricsException

Error closing connection to Graphite.

Error message

Error closing connection to Graphite.

What it means

GraphiteSink.flush() calls graphite.flush() every flush cycle. If the flush throws (broken connection), the sink logs WARN "Error flushing metrics to Graphite." and attempts graphite.close(); when the close also throws, MetricsException("Error closing connection to Graphite.", e1) is thrown. As with the putMetrics variant, the surfaced exception is the secondary close failure and the root network error is in the preceding WARN log.

Source

Thrown at hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/metrics2/sink/GraphiteSink.java:119

      LOG.warn("Error sending metrics to Graphite.", e);
      try {
        graphite.close();
      } catch (Exception e1) {
        throw new MetricsException("Error closing connection to Graphite", e1);
      }
    }
  }

  @Override
  public void flush() {
    try {
      graphite.flush();
    } catch (Exception e) {
      LOG.warn("Error flushing metrics to Graphite.", e);
      try {
        graphite.close();
      } catch (Exception e1) {
        throw new MetricsException("Error closing connection to Graphite.", e1);
      }
    }
  }

  @Override
  public void close() throws IOException {
    graphite.close();
  }

  public static class Graphite {
    private final static int MAX_CONNECTION_FAILURES = 5;

    private String serverHost;
    private int serverPort;
    private Writer writer = null;
    private Socket socket = null;
    private int connectionFailures = 0;

View on GitHub (pinned to 2add963021)

Solutions

  1. Check the WARN 'Error flushing metrics to Graphite.' line for the underlying exception
  2. Restore Graphite server availability and verify server_host/server_port connectivity (nc -vz host port)
  3. Rely on lazy reconnection: the next putMetrics/write cycle re-establishes the socket
  4. Restart the daemon after remediation if the sink has hit its 5-failure silent-give-up limit
Defensive patterns

Strategy: retry

Validate before calling

try (Socket s = new Socket()) {
  s.connect(new InetSocketAddress(graphiteHost, graphitePort), 3000);
} catch (IOException e) {
  LOG.warn("Graphite endpoint unhealthy before flush: {}", e.toString());
}

Try / catch

try {
  sink.flush();
} catch (MetricsException e) {
  // flush + close both failed; connection is dead but the next write() reconnects
  LOG.warn("Graphite flush/close failed; socket will be re-established on next write", e);
}

Prevention

When it happens

Trigger: flush() on a socket already broken by a Graphite restart, connection reset, or idle-timeout firewall drop; the subsequent close() on the dead socket throws as well.

Common situations: Graphite server restarts; LB/NAT idle timeouts shorter than the sink flush interval; network partitions that leave the TCP connection half-open.

Related errors


AI-assisted analysis of apache/hadoop@2add963021 (2026-08-22). Data as JSON: /api/errors/407387e651001ec5. Report an issue: GitHub.