{"record":{"id":"16fa167afe9b34d8","repo":"apache/hadoop","slug":"shuffleprovider-service-was-not-found-in-the-l","errorCode":null,"errorMessage":"ShuffleProvider Service: {} was NOT found in the list of aux-services that are available in this NM. You may need to specify this ShuffleProvider as an aux-service in your yarn-site.xml","messagePattern":"ShuffleProvider Service: (.+?) was NOT found in the list of aux-services that are available in this NM\\. You may need to specify this ShuffleProvider as an aux-service in your yarn-site\\.xml","errorType":"exception","errorClass":"YarnRuntimeException","httpStatus":null,"severity":"error","filePath":"hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app/src/main/java/org/apache/hadoop/mapreduce/v2/app/job/impl/TaskAttemptImpl.java","lineNumber":1105,"sourceCode":"          conf.getStringCollection(YarnConfiguration.NM_AUX_SERVICES);\n\n      for (final String shuffleProvider : shuffleProviders) {\n        if (shuffleProvider\n            .equals(ShuffleHandler.MAPREDUCE_SHUFFLE_SERVICEID)) {\n          continue; // skip built-in shuffle-provider that was already inserted\n                    // with shuffle secret key\n        }\n        if (auxNames.contains(shuffleProvider)) {\n          LOG.info(\"Adding ShuffleProvider Service: \" + shuffleProvider\n              + \" to serviceData\");\n          // This only serves for INIT_APP notifications\n          // The shuffle service needs to be able to work with the host:port\n          // information provided by the AM\n          // (i.e. shuffle services which require custom location / other\n          // configuration are not supported)\n          serviceData.put(shuffleProvider, ByteBuffer.allocate(0));\n        } else {\n          throw new YarnRuntimeException(\"ShuffleProvider Service: \"\n              + shuffleProvider\n              + \" was NOT found in the list of aux-services that are \"\n              + \"available in this NM. You may need to specify this \"\n              + \"ShuffleProvider as an aux-service in your yarn-site.xml\");\n        }\n      }\n    }\n  }\n\n  static ContainerLaunchContext createContainerLaunchContext(\n      Map<ApplicationAccessType, String> applicationACLs,\n      Configuration conf, Token<JobTokenIdentifier> jobToken, Task remoteTask,\n      final org.apache.hadoop.mapred.JobID oldJobId,\n      WrappedJvmID jvmID,\n      TaskAttemptListener taskAttemptListener,\n      Credentials credentials) {\n\n    synchronized (commonContainerSpecLock) {","sourceCodeStart":1087,"sourceCodeEnd":1123,"githubUrl":"https://github.com/apache/hadoop/blob/2add9630210752f88ceb1bb74eb65e37bf41da8e/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app/src/main/java/org/apache/hadoop/mapreduce/v2/app/job/impl/TaskAttemptImpl.java#L1087-L1123","documentation":"addExternalShuffleProviders throws YarnRuntimeException at container-launch setup when a name listed in mapreduce.job.shuffle.provider.services is not present in yarn.nodemanager.aux-services as known to the AM. Before launching task containers, the AM writes serviceData for each configured external shuffle provider; a provider missing from the NM's aux-service list cannot receive it, so the job is failed rather than silently shuffling wrong.","triggerScenarios":"Job sets mapreduce.job.shuffle.provider.services=mapred_shufflex but NMs only run mapreduce_shuffle; the AM's view of yarn.nodemanager.aux-services (mirrored from cluster config) lacks the provider; only some NMs were restarted after adding the service.","commonSituations":"Rolling out a new aux-service (e.g., an SSD/offheap shuffle plugin) where yarn-site.xml was updated on the job client but not on NMs; typo between the job config name and the aux-services entry; heterogeneous cluster after partial restart.","solutions":["Add the provider to yarn.nodemanager.aux-services (plus its yarn.nodemanager.aux-services.<name>.class) in the NM yarn-site.xml on every NodeManager and restart them","Verify spelling matches exactly between mapreduce.job.shuffle.provider.services and the aux-services entry","Until rollout completes, remove the provider from the job's shuffle-provider list"],"exampleFix":"<!-- before: job asks for a provider NMs don't run -->\n<property><name>mapreduce.job.shuffle.provider.services</name><value>mapred_shufflex</value></property>\n\n<!-- after: NM yarn-site.xml on all nodes, then restart NM -->\n<property><name>yarn.nodemanager.aux-services</name><value>mapreduce_shuffle,mapred_shufflex</value></property>\n<property><name>yarn.nodemanager.aux-services.mapred_shufflex.class</name><value>com.example.ShufflexService</value></property>","handlingStrategy":"validation","validationCode":"// preflight: every configured shuffle provider must be an NM aux-service\nCollection<String> providers = conf.getStringCollection(\"mapreduce.job.shuffle.provider.services\");\nproviders.remove(\"mapreduce_shuffle\"); // built-in, always present\nCollection<String> aux = conf.getStringCollection(\"yarn.nodemanager.aux-services\");\nif (!aux.containsAll(providers)) {\n  throw new IllegalArgumentException(\"Missing aux-services: \" +\n      Sets.difference(new HashSet<>(providers), new HashSet<>(aux)));\n}","typeGuard":null,"tryCatchPattern":null,"preventionTips":["Roll out new aux-services to all NMs (and restart them) before jobs reference them","Keep the job's mapreduce.job.shuffle.provider.services list spelled exactly like the aux-services entries","After NM config changes, verify with 'yarn node -status' that the aux-service shows up before submitting"],"tags":["mapreduce","shuffle","aux-services","yarn","configuration"],"backgroundTag":"aux-service-missing","analyzedSha":"2add9630210752f88ceb1bb74eb65e37bf41da8e","analyzedAt":"2026-08-22T19:55:07.957Z","schemaVersion":2},"datasetVersion":"2026-08-22T20:17:22.307Z"}