{"record":{"id":"83b4a261e1fb4eaf","repo":"apache/hadoop","slug":"exception-while-initializing-metric-credentials","errorCode":null,"errorMessage":"Exception while initializing metric credentials ","messagePattern":"Exception while initializing metric credentials ","errorType":"exception","errorClass":"IOException","httpStatus":null,"severity":"error","filePath":"hadoop-tools/hadoop-azure/src/main/java/org/apache/hadoop/fs/azurebfs/services/AbfsMetricsManager.java","lineNumber":153,"sourceCode":"        String metricAccountKey = abfsConfiguration.getMetricAccountKey();\n        this.metricFormat = abfsConfiguration.getMetricFormat();\n        if (isNotEmpty(metricAccountName) && isNotEmpty(\n            metricAccountKey)) {\n          int dotIndex = metricAccountName.indexOf(AbfsHttpConstants.DOT);\n          if (dotIndex <= 0) {\n            throw new InvalidUriException(\n                metricAccountName + \" - account name is not fully qualified.\");\n          }\n          try {\n            metricSharedkeyCredentials = new SharedKeyCredentials(\n                metricAccountName.substring(0, dotIndex),\n                metricAccountKey);\n            hasSeparateMetricAccount = true;\n            setMetricsUrl(metricAccountName.startsWith(HTTPS_SCHEME)\n                ? metricAccountName : HTTPS_SCHEME + COLON\n                + FORWARD_SLASH + FORWARD_SLASH + metricAccountName);\n          } catch (IllegalArgumentException e) {\n            throw new IOException(\n                \"Exception while initializing metric credentials \", e);\n          }\n        } else {\n          setMetricsUrl(baseUrlString.substring(0, indexLastForwardSlash + 1));\n        }\n        // Once the metric URL is set, initialize the metrics\n        abfsCounters.initializeMetrics(metricFormat, abfsConfiguration);\n        // Metrics emitter scheduler\n        this.metricsEmitScheduler\n            = Executors.newSingleThreadScheduledExecutor();\n        // run every 1 minute to check the metrics count\n        this.metricsEmitScheduler.scheduleWithFixedDelay(\n            () -> {\n              if (abfsCounters.getAbfsBackoffMetrics()\n                  .getMetricValue(TOTAL_NUMBER_OF_REQUESTS)\n                  >= abfsConfiguration.getMetricsEmitThreshold()) {\n                emitCollectedMetrics();\n              }","sourceCodeStart":135,"sourceCodeEnd":171,"githubUrl":"https://github.com/apache/hadoop/blob/2add9630210752f88ceb1bb74eb65e37bf41da8e/hadoop-tools/hadoop-azure/src/main/java/org/apache/hadoop/fs/azurebfs/services/AbfsMetricsManager.java#L135-L171","documentation":"AbfsMetricsManager wraps the SharedKeyCredentials constructor in a try/catch and rethrows IllegalArgumentException as IOException(\"Exception while initializing metric credentials \", e). SharedKeyCredentials throws IllegalArgumentException when the key is not valid Base64, so this is almost always a malformed fs.azure.metrics.account.key. It fires only when both metrics account name and key are set and metrics collection is enabled.","triggerScenarios":"fs.azure.metrics.account.key containing a non-Base64 string: wrong key, truncated copy/paste, wrapped in quotes/whitespace, a SAS token pasted instead of the account key, or a key from a different (new-style) storage account whose key format does not parse.","commonSituations":"Secrets mangled by config management (extra newline, XML entity escaping, template placeholders); rotating keys and pasting the connection string instead of the key; CI configs injecting the wrong secret variable.","solutions":["Paste the exact Base64 access key from the Azure portal (Storage account > Access keys) into fs.azure.metrics.account.key","Verify locally: Base64.getDecoder().decode(key) must succeed without exception","Check for surrounding whitespace/quotes/newlines introduced by config files or secret managers","If a separate metrics account is not needed, unset both metrics account name and key"],"exampleFix":"# before (shell-injected, whitespace + wrong secret type)\nfs.azure.metrics.account.key=\" AccountKey='sv=2020-...'\"   # SAS token, not a key\n\n# after\nfs.azure.metrics.account.key=<exact base64 key, no quotes or spaces>","handlingStrategy":"validation","validationCode":"// Prove the key is valid Base64 before the FS tries to use it\nString key = conf.get(\"fs.azure.metrics.account.key\");\nif (key != null) {\n  try {\n    java.util.Base64.getDecoder().decode(key.trim());\n  } catch (IllegalArgumentException e) {\n    throw new IllegalArgumentException(\n        \"fs.azure.metrics.account.key is not valid Base64\", e);\n  }\n}","typeGuard":null,"tryCatchPattern":"try {\n  FileSystem fs = path.getFileSystem(conf);\n} catch (IOException e) {\n  if (e.getMessage() != null && e.getMessage().startsWith(\n      \"Exception while initializing metric credentials\")) {\n    // key is malformed: fix the secret, then re-create the FileSystem\n  } else throw e;\n}","preventionTips":["Copy access keys exactly from the Azure portal; never a connection string or SAS","Trim whitespace/newlines when injecting keys from secret managers","Test-decode the key in CI config validation"],"tags":["azure-blob","abfs","metrics","credentials","configuration","base64","hadoop"],"backgroundTag":"invalid-credentials","analyzedSha":"2add9630210752f88ceb1bb74eb65e37bf41da8e","analyzedAt":"2026-08-22T19:55:07.957Z","schemaVersion":2},"datasetVersion":"2026-08-22T20:17:22.307Z"}