apache/hadoop · error · IllegalArgumentException

Only one of the following keys can be specified for a single

Error message

Only one of the following keys can be specified for a single job: memory-mb, memory

What it means

While building the MR ApplicationMaster's resource requests, YARNRunner.generateResourceRequests calls ResourceUtils.getRequestedResourcesFromConfig over the yarn.app.mapreduce.am.resource.* prefix. 'memory-mb' and 'memory' are two config spellings for the same memory resource; the loop tracks a memorySet flag and throws IllegalArgumentException as soon as the second alias appears, because the two keys would compete for the same Resource field.

Source

Thrown at hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-jobclient/src/main/java/org/apache/hadoop/mapred/YARNRunner.java:687

      appContext.setPriority(Priority.newInstance(iPriority));
    }

    return appContext;
  }

  private List<ResourceRequest> generateResourceRequests() throws IOException {
    Resource capability = recordFactory.newRecordInstance(Resource.class);
    boolean memorySet = false;
    boolean cpuVcoresSet = false;
    List<ResourceInformation> resourceRequests = ResourceUtils
        .getRequestedResourcesFromConfig(conf, MR_AM_RESOURCE_PREFIX);
    for (ResourceInformation resourceReq : resourceRequests) {
      String resourceName = resourceReq.getName();
      if (MRJobConfig.RESOURCE_TYPE_NAME_MEMORY.equals(resourceName) ||
          MRJobConfig.RESOURCE_TYPE_ALTERNATIVE_NAME_MEMORY.equals(
              resourceName)) {
        if (memorySet) {
          throw new IllegalArgumentException(
              "Only one of the following keys " +
                  "can be specified for a single job: " +
                  MRJobConfig.RESOURCE_TYPE_ALTERNATIVE_NAME_MEMORY + ", " +
                  MRJobConfig.RESOURCE_TYPE_NAME_MEMORY);
        }
        String units = isEmpty(resourceReq.getUnits()) ?
            ResourceUtils.getDefaultUnit(ResourceInformation.MEMORY_URI) :
              resourceReq.getUnits();
        capability.setMemorySize(
            UnitsConversionUtil.convert(units, "Mi", resourceReq.getValue()));
        memorySet = true;
        if (conf.get(MRJobConfig.MR_AM_VMEM_MB) != null) {
          LOG.warn("Configuration " + MR_AM_RESOURCE_PREFIX +
              resourceName + "=" + resourceReq.getValue() +
              resourceReq.getUnits() + " is overriding the " +
              MRJobConfig.MR_AM_VMEM_MB + "=" +
              conf.get(MRJobConfig.MR_AM_VMEM_MB) + " configuration");
        }

View on GitHub (pinned to 2add963021)

Solutions

Remove one of the two keys from mapred-site.xml / job Configuration; keep the canonical 'yarn.app.mapreduce.am.resource.memory' form
If a -D CLI override introduced the duplicate, drop the site-file copy instead
Dump the effective config (e.g. 'hadoop job -conf ...' or conf.get on both keys in a dry run) to find where each alias enters the configuration chain

Example fix

# before (mapred-site.xml carries both)
<property><name>yarn.app.mapreduce.am.resource.memory-mb</name><value>2048</value></property>
<property><name>yarn.app.mapreduce.am.resource.memory</name><value>2048</value></property>

# after (single canonical key)
<property><name>yarn.app.mapreduce.am.resource.memory</name><value>2048</value></property>

Defensive patterns

Strategy: validation

Validate before calling

if (conf.get("yarn.app.mapreduce.am.resource.memory-mb") != null
    && conf.get("yarn.app.mapreduce.am.resource.memory") != null) {
  throw new IllegalArgumentException(
      "Both memory-mb and memory set for the AM; remove one (keep 'memory')");
}

Try / catch

try {
  jobClient.submitJob(jobConf);
} catch (IllegalArgumentException e) {
  if (e.getMessage().contains("memory-mb")) {
    jobConf.unset("yarn.app.mapreduce.am.resource.memory-mb"); // keep canonical key
    jobClient.submitJob(jobConf);
  } else { throw e; }
}

Prevention

Standardize on the canonical 'yarn.app.mapreduce.am.resource.<type>' spelling in all templates
Audit merged configurations (site files + -D overrides) for duplicate resource aliases before submission
In CI, assert that at most one memory alias is present in generated job configs

When it happens

Trigger: mapred-site.xml (or a -D override) contains both yarn.app.mapreduce.am.resource.memory-mb and yarn.app.mapreduce.am.resource.memory, and a job is submitted through YARNRunner. The exception is thrown during submitJob, before any container is requested.

Common situations: Merging job configs from multiple templates where one uses the legacy 'memory-mb' spelling and newer code sets 'memory'; site files upgraded across Hadoop versions accumulating both keys; automation that sets memory programmatically on top of a site file that already defines the alias.

Related errors

AI-assisted analysis of apache/hadoop@2add963021 (2026-08-22). Data as JSON: /api/errors/448ad9fbc72bd804. Report an issue: GitHub.