{"record":{"id":"e7cbd6c3fb042b97","repo":"apache/hadoop","slug":"error-writing-metric-to-statsd","errorCode":null,"errorMessage":"Error writing metric to StatsD","messagePattern":"Error writing metric to StatsD","errorType":"exception","errorClass":"MetricsException","httpStatus":null,"severity":"warning","filePath":"hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/metrics2/sink/StatsDSink.java","lineNumber":155,"sourceCode":"      }\n      StringBuilder line = new StringBuilder();\n      line.append(buf.toString())\n          .append(metric.name().replace(' ', '_'))\n          .append(\":\")\n          .append(metric.value())\n          .append(\"|\")\n          .append(type);\n      writeMetric(line.toString());\n    }\n\n  }\n\n  public void writeMetric(String line) {\n    try {\n      statsd.write(line);\n    } catch (IOException e) {\n      LOG.warn(\"Error sending metrics to StatsD\", e);\n      throw new MetricsException(\"Error writing metric to StatsD\", e);\n    }\n  }\n\n  @Override\n  public void flush() {\n  }\n\n  @Override\n  public void close() throws IOException {\n    statsd.close();\n  }\n\n  /**\n   * Class that sends UDP packets to StatsD daemon.\n   *\n   */\n  public static class StatsD {\n","sourceCodeStart":137,"sourceCodeEnd":173,"githubUrl":"https://github.com/apache/hadoop/blob/2add9630210752f88ceb1bb74eb65e37bf41da8e/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/metrics2/sink/StatsDSink.java#L137-L173","documentation":"StatsDSink sends each metric as one UDP datagram ('host.service.context.name.metricname:value|type'). writeMetric catches IOException from statsd.write and rethrows it as MetricsException(\"Error writing metric to StatsD\") after logging WARN 'Error sending metrics to StatsD'. The IOException originates in createSocket (building the InetSocketAddress for server.host; failures are wrapped via NetUtils.wrapException) or in DatagramSocket.send. Because the transport is UDP, a merely stopped StatsD server does NOT produce this error — it indicates name-resolution or socket-level problems.","triggerScenarios":"*.sink.statsd.server.host does not resolve (DNS typo, NXDOMAIN) so createSocket throws UnknownHostException; the datagram socket is closed concurrently (close() nulls it, so the next write re-creates it lazily); a send() IOException from the OS.","commonSituations":"Typo'd or short (unqualified) statsd hostname in hadoop-metrics2.properties; DNS outages in containers; the sink restarted after close() with stale host config.","solutions":["Verify resolution from the daemon host: getent hosts <server.host> / nslookup <server.host>","Fix *.sink.statsd.server.host / *.sink.statsd.server.port values","Recovery is automatic once DNS is fixed — write() re-creates the socket when it is null","Remember UDP gives no feedback if the server is merely down; do not chase this error for that case"],"exampleFix":"# before\n*.sink.statsd.server.host=statsd  # unresolvable bare name\n*.sink.statsd.server.port=8125\n\n# after\n*.sink.statsd.server.host=statsd.prod.example.com\n*.sink.statsd.server.port=8125","handlingStrategy":"retry","validationCode":"InetSocketAddress addr = new InetSocketAddress(serverHost, serverPort);\nif (addr.isUnresolved()) {\n  throw new IllegalStateException(\"StatsD host does not resolve: \" + serverHost\n      + \" — fix *.sink.statsd.server.host before deploy\");\n}","typeGuard":null,"tryCatchPattern":"try {\n  sink.writeMetric(line);\n} catch (MetricsException e) {\n  // UDP transport: thrown for DNS/socket problems (NetUtils.wrapException cause), not for a down server\n  LOG.warn(\"StatsD write failed ({}); socket re-created lazily on next write\",\n      e.getCause() == null ? e : e.getCause().toString());\n}","preventionTips":["Resolve-verify the statsd hostname from each node class (getent hosts) before shipping config","Use fully qualified hostnames in *.sink.statsd.server.host to avoid search-domain surprises in containers","Remember UDP is fire-and-forget: a stopped StatsD never raises this error, so monitor the StatsD side separately","Set *.sink.statsd.server.port explicitly (default 8125) since a missing value fails earlier as NumberFormatException"],"tags":["statsd","metrics2","udp","dns","network"],"backgroundTag":"dns-resolution-failed","analyzedSha":"2add9630210752f88ceb1bb74eb65e37bf41da8e","analyzedAt":"2026-08-22T19:55:07.957Z","schemaVersion":2},"datasetVersion":"2026-08-22T20:17:22.307Z"}