{"record":{"id":"0c4ed91327d92332","repo":"apache/hadoop","slug":"unable-to-map-logical-nameservice-uri-to-a-na","errorCode":null,"errorMessage":"Unable to map logical nameservice URI '{}' to a NameNode. Local configuration does not have a failover proxy provider configured.","messagePattern":"Unable to map logical nameservice URI '(.+?)' to a NameNode\\. Local configuration does not have a failover proxy provider configured\\.","errorType":"exception","errorClass":"IOException","httpStatus":null,"severity":"error","filePath":"hadoop-hdfs-project/hadoop-hdfs-client/src/main/java/org/apache/hadoop/hdfs/DFSClient.java","lineNumber":861,"sourceCode":"        nn.cancelDelegationToken(delToken);\n      } catch (RemoteException re) {\n        throw re.unwrapRemoteException(InvalidToken.class,\n            AccessControlException.class);\n      }\n    }\n\n    private static ClientProtocol getNNProxy(\n        Token<DelegationTokenIdentifier> token, Configuration conf)\n        throws IOException {\n      URI uri = HAUtilClient.getServiceUriFromToken(\n          HdfsConstants.HDFS_URI_SCHEME, token);\n      if (HAUtilClient.isTokenForLogicalUri(token) &&\n          !HAUtilClient.isLogicalUri(conf, uri)) {\n        // If the token is for a logical nameservice, but the configuration\n        // we have disagrees about that, we can't actually renew it.\n        // This can be the case in MR, for example, if the RM doesn't\n        // have all of the HA clusters configured in its configuration.\n        throw new IOException(\"Unable to map logical nameservice URI '\" +\n            uri + \"' to a NameNode. Local configuration does not have \" +\n            \"a failover proxy provider configured.\");\n      }\n\n      ProxyAndInfo<ClientProtocol> info =\n          NameNodeProxiesClient.createProxyWithClientProtocol(conf, uri, null);\n      assert info.getDelegationTokenService().equals(token.getService()) :\n          \"Returned service '\" + info.getDelegationTokenService().toString() +\n              \"' doesn't match expected service '\" +\n              token.getService().toString() + \"'\";\n\n      return info.getProxy();\n    }\n\n    @Override\n    public boolean isManaged(Token<?> token) throws IOException {\n      return true;\n    }","sourceCodeStart":843,"sourceCodeEnd":879,"githubUrl":"https://github.com/apache/hadoop/blob/2add9630210752f88ceb1bb74eb65e37bf41da8e/hadoop-hdfs-project/hadoop-hdfs-client/src/main/java/org/apache/hadoop/hdfs/DFSClient.java#L843-L879","documentation":"To renew/cancel a delegation token whose service is a logical HA nameservice URI (e.g. hdfs://mycluster), DFSClient must build a failover proxy from local configuration. If the local config has no failover proxy provider for that nameservice (missing dfs.client.failover.proxy.provider.<ns> and dfs.ha.namenodes.<ns>), it cannot reach any NameNode and throws this IOException. The source comment cites exactly this MR scenario: a ResourceManager that lacks other clusters' HA config.","triggerScenarios":"A component (YARN RM, MR job client, Oozie, History Server) renewing a token minted by cluster B while its own hdfs-site.xml only configures cluster A; distcp or federation setups where tokens cross cluster boundaries; minimal-config daemons handed foreign tokens.","commonSituations":"Multi-HA-cluster enterprises where RMs renew job tokens on behalf of jobs from other clusters; onboarding a new HA namespace without rolling its client config to all services; token-based workflows (Oozie coordinators) spanning trust domains.","solutions":["Add the missing nameservice's client HA config to the renewing component: dfs.ha.namenodes.<ns>, dfs.client.failover.proxy.provider.<ns> (ConfiguredFailoverProxyProvider), and dfs.namenode.rpc-address.<ns>.<nn> for each NameNode.","Distribute the union of all clusters' HDFS client configs to every host/service that may renew tokens (config management or YARN's config delivery).","Validate before renewing: parse the token's service URI host and check the provider key exists in conf; fail fast with a clear config-oriented message.","As a design alternative, let the RM renew with its own keytab instead of renewing borrowed tokens it cannot resolve."],"exampleFix":"// before: token service is hdfs://ns2 but local config only knows ns1\nclient.renewDelegationToken(token);\n// IOException: Unable to map logical nameservice URI 'hdfs://ns2' ...\n\n// after: add to the renewing component's hdfs-site.xml\n// <property><name>dfs.ha.namenodes.ns2</name><value>nn1,nn2</value></property>\n// <property><name>dfs.client.failover.proxy.provider.ns2</name>\n//   <value>org.apache.hadoop.hdfs.server.namenode.ha.ConfiguredFailoverProxyProvider</value></property>\n// <property><name>dfs.namenode.rpc-address.ns2.nn1</name><value>host1:8020</value></property>\n// <property><name>dfs.namenode.rpc-address.ns2.nn2</name><value>host2:8020</value></property>","handlingStrategy":"validation","validationCode":"// fail fast if local config cannot resolve the token's logical nameservice\nURI uri = HAUtilClient.getServiceUriFromToken(HdfsConstants.HDFS_URI_SCHEME, token);\nif (HAUtilClient.isTokenForLogicalUri(token)) {\n  String ns = uri.getHost();\n  if (conf.get(\"dfs.client.failover.proxy.provider.\" + ns) == null) {\n    throw new IOException(\"No failover proxy provider configured for nameservice \" + ns);\n  }\n}","typeGuard":null,"tryCatchPattern":"catch (IOException e) {\n  if (e.getMessage().contains(\"failover proxy provider\")) {\n    // config gap: report which nameservice is missing HA entries instead of retrying\n  }\n  throw e;\n}","preventionTips":["Ship the union of all HA clusters' client configs to every service that renews tokens.","Add a startup lint: for each known nameservice, assert dfs.ha.namenodes.<ns> and the provider key exist.","Prefer RM-side keytab renewal designs over renewing borrowed foreign tokens."],"tags":["hdfs","high-availability","delegation-token","configuration","multi-cluster","token-renewal"],"backgroundTag":"ha-failover-proxy-missing","analyzedSha":"2add9630210752f88ceb1bb74eb65e37bf41da8e","analyzedAt":"2026-08-22T19:55:07.957Z","schemaVersion":2},"datasetVersion":"2026-08-22T20:17:22.307Z"}