{"record":{"id":"1e7404c742b8702c","repo":"apache/hadoop","slug":"metrics2-configuration-is-missing-key-property","errorCode":null,"errorMessage":"Metrics2 configuration is missing ${key} property","messagePattern":"Metrics2 configuration is missing (.+?) property","errorType":"exception","errorClass":"MetricsException","httpStatus":null,"severity":"error","filePath":"hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/metrics2/sink/RollingFileSystemSink.java","lineNumber":426,"sourceCode":"  private long getNonNegative(String key, int defaultValue) {\n    int flushOffsetIntervalMillis = properties.getInt(key, defaultValue);\n\n    if (flushOffsetIntervalMillis < 0) {\n      throw new MetricsException(\"The \" + key + \" property must be \"\n          + \"non-negative. Value was \" + flushOffsetIntervalMillis);\n    }\n\n    return flushOffsetIntervalMillis;\n  }\n\n  /**\n   * Throw a {@link MetricsException} if the given property is not set.\n   *\n   * @param key the key to validate\n   */\n  private void checkIfPropertyExists(String key) {\n    if (!properties.containsKey(key)) {\n      throw new MetricsException(\"Metrics2 configuration is missing \" + key\n          + \" property\");\n    }\n  }\n\n  /**\n   * Return the supplied configuration for testing or otherwise load a new\n   * configuration.\n   *\n   * @return the configuration to use\n   */\n  private Configuration loadConf() {\n    Configuration c;\n\n    if (suppliedConf != null) {\n      c = suppliedConf;\n    } else {\n      // The config we're handed in init() isn't the one we want here, so we\n      // create a new one to pick up the full settings.","sourceCodeStart":408,"sourceCodeEnd":444,"githubUrl":"https://github.com/apache/hadoop/blob/2add9630210752f88ceb1bb74eb65e37bf41da8e/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/metrics2/sink/RollingFileSystemSink.java#L408-L444","documentation":"With Kerberos security enabled, RollingFileSystemSink.init() requires two extra sink properties: keytab-key and principal-key, whose VALUES are names of Configuration keys that hold the actual keytab path and principal (e.g. yarn.nodemanager.keytab / yarn.nodemanager.principal). checkIfPropertyExists throws MetricsException(\"Metrics2 configuration is missing <key> property\") naming exactly which of the two is absent from the sink's configuration subset.","triggerScenarios":"Kerberized cluster with the rolling sink configured, but <prefix>.sink.<instance>.keytab-key or .principal-key (or both) omitted from hadoop-metrics2.properties.","commonSituations":"Enabling security on an existing cluster and adding the rolling sink with only basepath/roll-interval; renaming the sink instance so old property prefixes no longer match (SubsetConfiguration keys are prefix-dependent).","solutions":["Add both properties, pointing at existing Configuration keys: <prefix>.sink.<instance>.keytab-key=dfs.namenode.keytab.file and .principal-key=dfs.namenode.kerberos.principal","Verify the referenced keys exist in the effective (merged, XML) Configuration and are not filtered out","Confirm the property prefix matches the sink instance name exactly"],"exampleFix":"# before (secure cluster)\nnamenode.sink.rolling.class=org.apache.hadoop.metrics2.sink.RollingFileSystemSink\n# -> Metrics2 configuration is missing keytab-key property\n\n# after\nnamenode.sink.rolling.class=org.apache.hadoop.metrics2.sink.RollingFileSystemSink\nnamenode.sink.rolling.keytab-key=dfs.namenode.keytab.file\nnamenode.sink.rolling.principal-key=dfs.namenode.kerberos.principal","handlingStrategy":"validation","validationCode":"if (UserGroupInformation.isSecurityEnabled()) {\n  for (String key : new String[] {\"keytab-key\", \"principal-key\"}) {\n    if (!subsetConf.containsKey(key)) {\n      throw new IllegalStateException(\"Secure cluster: sink config missing \" + key\n          + \" (value must name a Configuration key holding the \"\n          + (key.equals(\"keytab-key\") ? \"keytab path\" : \"principal\") + \")\");\n    }\n  }\n}","typeGuard":null,"tryCatchPattern":"try {\n  sink.init(subsetConf);\n} catch (MetricsException e) {\n  // 'Metrics2 configuration is missing <key> property' — add keytab-key and/or principal-key\n  LOG.error(\"Add the missing secure-login sink property: {}\", e.getMessage());\n}","preventionTips":["On kerberized clusters always ship both keytab-key and principal-key with every rolling sink instance","Keep the property prefix (<prefix>.sink.<instance>.) in sync with the sink instance name","Automate hadoop-metrics2.properties checks in config management (puppet/ansible) to catch missing keys"],"tags":["kerberos","security","metrics2","rolling-file-sink","missing-config"],"backgroundTag":"missing-config-property","analyzedSha":"2add9630210752f88ceb1bb74eb65e37bf41da8e","analyzedAt":"2026-08-22T19:55:07.957Z","schemaVersion":2},"datasetVersion":"2026-08-23T01:17:44.959Z"}