{"record":{"id":"91252b0d17906035","repo":"apache/hadoop","slug":"failed-to-putmetrics","errorCode":null,"errorMessage":"Failed to putMetrics","messagePattern":"Failed to putMetrics","errorType":"exception","errorClass":"MetricsException","httpStatus":null,"severity":"error","filePath":"hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/metrics2/sink/ganglia/GangliaSink30.java","lineNumber":191,"sourceCode":"            // slope\n            metric.visit(gangliaMetricVisitor);\n            type = gangliaMetricVisitor.getType();\n            slopeFromMetric = gangliaMetricVisitor.getSlope();\n\n            GangliaConf gConf = getGangliaConfForMetric(name);\n            calculatedSlope = calculateSlope(gConf, slopeFromMetric);\n\n            // send metric to Ganglia\n            emitMetric(groupName, name, type, metric.value().toString(), gConf,\n                calculatedSlope);\n\n            // reset the length of the buffer for next iteration\n            sb.setLength(sbBaseLen);\n          }\n        }\n      }\n    } catch (IOException io) {\n      throw new MetricsException(\"Failed to putMetrics\", io);\n    }\n  }\n\n  // Calculate the slope from properties and metric\n  private GangliaSlope calculateSlope(GangliaConf gConf,\n      GangliaSlope slopeFromMetric) {\n    if (gConf.getSlope() != null) {\n      // if slope has been specified in properties, use that\n      return gConf.getSlope();\n    } else if (slopeFromMetric != null) {\n      // slope not specified in properties, use derived from Metric\n      return slopeFromMetric;\n    } else {\n      return DEFAULT_SLOPE;\n    }\n  }\n\n  /**","sourceCodeStart":173,"sourceCodeEnd":209,"githubUrl":"https://github.com/apache/hadoop/blob/2add9630210752f88ceb1bb74eb65e37bf41da8e/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/metrics2/sink/ganglia/GangliaSink30.java#L173-L209","documentation":"MetricsException('Failed to putMetrics') thrown by GangliaSink30.putMetrics() wrapping any IOException raised while building and emitting metric datagrams (typically from emitToGangliaHosts sending to the gmond collector). The original IOException is preserved as the cause, so the real network failure (send error, unresolved host, closed socket) is visible via getCause(). The metrics2 framework catches MetricsException in MetricsSinkAdapter, logs it, and continues, so it usually appears as a recurring error line rather than a crash.","triggerScenarios":"datagramSocket.send() failing with network-unreachable / socket-closed; an unresolved ganglia host raising UnknownHostException inside emitToGangliaHosts; gmond not running or not listening on the configured UDP port so ICMP port-unreachable is reflected back to the sender.","commonSituations":"gmond (Ganglia monitoring daemon) down or restarted; firewall between the Hadoop node and the ganglia collector blocking UDP 8649; wrong port in the servers property; GangliaSink30 multicast configuration pointing at a bad multicast channel.","solutions":["Read the cause: catch MetricsException and inspect getCause() — the wrapped IOException states the real failure","Verify gmond is listening on the configured UDP port on the target: 'nc -u -v <gangliaHost> 8649' or check 'ss -lun' on the collector","Check firewall rules allow outbound UDP to the ganglia port from every Hadoop node","Confirm the '<prefix>.sink.ganglia.servers' host:port values and that the host resolves (see UnknownHostException 'Unresolved host' from the same sink)"],"exampleFix":"// before\nsink.putMetrics(record);  // throws MetricsException(\"Failed to putMetrics\", io) every flush cycle\n\n// after\n// fix the collector side: start gmond on 10.0.0.5:8649 (systemctl start gmond)\n// and validate reachability: nc -u -vz 10.0.0.5 8649","handlingStrategy":"try-catch","validationCode":"// Check UDP reachability before enabling the sink in production\n// nc -u -vz <gangliaHost> 8649  (shell-level check, not in-process)","typeGuard":null,"tryCatchPattern":"try {\n  sink.putMetrics(record);\n} catch (MetricsException e) {\n  Throwable cause = e.getCause(); // the real IOException\n  LOG.warn(\"Ganglia emit failed: {}\", cause, cause instanceof UnknownHostException ? \"fix servers property\" : \"check gmond/network\");\n}","preventionTips":["Monitor for recurring 'Failed to putMetrics' in daemon logs — it signals collector-side problems","Keep gmond monitored and the UDP port open in firewalls from every Hadoop node","Read getCause(), not the generic message, to classify DNS vs send failures"],"tags":["ganglia","metrics2","udp","io","hadoop-common"],"backgroundTag":"metrics-sink-send-failure","analyzedSha":"2add9630210752f88ceb1bb74eb65e37bf41da8e","analyzedAt":"2026-08-22T19:55:07.957Z","schemaVersion":2},"datasetVersion":"2026-08-22T20:17:22.307Z"}