apache/hadoop · error · IllegalArgumentException

Only one of the following keys can be specified for a…

Error message

Only one of the following keys can be specified for a single job: memory-mb, memory

What it means

While building the MR ApplicationMaster's resource requests, YARNRunner.generateResourceRequests calls ResourceUtils.getRequestedResourcesFromConfig over the yarn.app.mapreduce.am.resource.* prefix. 'memory-mb' and 'memory' are two config spellings for the same memory resource; the loop tracks a memorySet flag and throws IllegalArgumentException as soon as the second alias appears, because the two keys would compete for the same Resource field.

Solutions

Remove one of the two keys from mapred-site.xml / job Configuration; keep the canonical 'yarn.app.mapreduce.am.resource.memory' form
If a -D CLI override introduced the duplicate, drop the site-file copy instead
Dump the effective config (e.g. 'hadoop job -conf ...' or conf.get on both keys in a dry run) to find where each alias enters the configuration chain

Example fix

# before (mapred-site.xml carries both)
<property><name>yarn.app.mapreduce.am.resource.memory-mb</name><value>2048</value></property>
<property><name>yarn.app.mapreduce.am.resource.memory</name><value>2048</value></property>

# after (single canonical key)
<property><name>yarn.app.mapreduce.am.resource.memory</name><value>2048</value></property>

Defensive patterns

Strategy: validation

Validate before calling

if (conf.get("yarn.app.mapreduce.am.resource.memory-mb") != null
    && conf.get("yarn.app.mapreduce.am.resource.memory") != null) {
  throw new IllegalArgumentException(
      "Both memory-mb and memory set for the AM; remove one (keep 'memory')");
}

Try / catch

try {
  jobClient.submitJob(jobConf);
} catch (IllegalArgumentException e) {
  if (e.getMessage().contains("memory-mb")) {
    jobConf.unset("yarn.app.mapreduce.am.resource.memory-mb"); // keep canonical key
    jobClient.submitJob(jobConf);
  } else { throw e; }
}

Prevention

Standardize on the canonical 'yarn.app.mapreduce.am.resource.<type>' spelling in all templates
Audit merged configurations (site files + -D overrides) for duplicate resource aliases before submission
In CI, assert that at most one memory alias is present in generated job configs

When it happens

Trigger: mapred-site.xml (or a -D override) contains both yarn.app.mapreduce.am.resource.memory-mb and yarn.app.mapreduce.am.resource.memory, and a job is submitted through YARNRunner. The exception is thrown during submitJob, before any container is requested.

Common situations: Merging job configs from multiple templates where one uses the legacy 'memory-mb' spelling and newer code sets 'memory'; site files upgraded across Hadoop versions accumulating both keys; automation that sets memory programmatically on top of a site file that already defines the alias.

Related errors

AI-assisted analysis of apache/hadoop@2add963021 (2026-08-22). Data as JSON: /api/errors/448ad9fbc72bd804. Report an issue: GitHub.

Appendix: source

Thrown at hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-jobclient/src/main/java/org/apache/hadoop/mapred/YARNRunner.java:687

      appContext.setPriority(Priority.newInstance(iPriority));
    }

    return appContext;
  }

  private List<ResourceRequest> generateResourceRequests() throws IOException {
    Resource capability = recordFactory.newRecordInstance(Resource.class);
    boolean memorySet = false;
    boolean cpuVcoresSet = false;
    List<ResourceInformation> resourceRequests = ResourceUtils
        .getRequestedResourcesFromConfig(conf, MR_AM_RESOURCE_PREFIX);
    for (ResourceInformation resourceReq : resourceRequests) {
      String resourceName = resourceReq.getName();
      if (MRJobConfig.RESOURCE_TYPE_NAME_MEMORY.equals(resourceName) ||
          MRJobConfig.RESOURCE_TYPE_ALTERNATIVE_NAME_MEMORY.equals(
              resourceName)) {
        if (memorySet) {
          throw new IllegalArgumentException(
              "Only one of the following keys " +
                  "can be specified for a single job: " +
                  MRJobConfig.RESOURCE_TYPE_ALTERNATIVE_NAME_MEMORY + ", " +
                  MRJobConfig.RESOURCE_TYPE_NAME_MEMORY);
        }
        String units = isEmpty(resourceReq.getUnits()) ?
            ResourceUtils.getDefaultUnit(ResourceInformation.MEMORY_URI) :
              resourceReq.getUnits();
        capability.setMemorySize(
            UnitsConversionUtil.convert(units, "Mi", resourceReq.getValue()));
        memorySet = true;
        if (conf.get(MRJobConfig.MR_AM_VMEM_MB) != null) {
          LOG.warn("Configuration " + MR_AM_RESOURCE_PREFIX +
              resourceName + "=" + resourceReq.getValue() +
              resourceReq.getUnits() + " is overriding the " +
              MRJobConfig.MR_AM_VMEM_MB + "=" +
              conf.get(MRJobConfig.MR_AM_VMEM_MB) + " configuration");
        }

View on GitHub (pinned to 2add963021)