apache/hadoop · error · IllegalArgumentException

Only one of the following keys can be specified for a single

Error message

Only one of the following keys can be specified for a single job: {}, {}

What it means

TaskAttemptImpl resource parsing throws IllegalArgumentException when the job configuration defines memory twice under the same resource prefix — e.g. both mapreduce.map.resource.memory and mapreduce.map.resource.memory-mb (or the reduce variants under mapreduce.reduce.resource.). The two names alias the same resource ('memory' vs its alternative name 'memory-mb'), so the AM cannot decide which value wins and rejects the job.

Source

Thrown at hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app/src/main/java/org/apache/hadoop/mapreduce/v2/app/job/impl/TaskAttemptImpl.java:732

    String resourceTypePrefix =
        getResourceTypePrefix(taskType);
    boolean memorySet = false;
    boolean cpuVcoresSet = false;
    if (RESOURCE_REQUEST_CACHE.get(taskType) != null) {
      resourceCapability = RESOURCE_REQUEST_CACHE.get(taskType);
      return;
    }
    if (resourceTypePrefix != null) {
      List<ResourceInformation> resourceRequests =
          ResourceUtils.getRequestedResourcesFromConfig(conf,
              resourceTypePrefix);
      for (ResourceInformation resourceRequest : resourceRequests) {
        String resourceName = resourceRequest.getName();
        if (MRJobConfig.RESOURCE_TYPE_NAME_MEMORY.equals(resourceName) ||
            MRJobConfig.RESOURCE_TYPE_ALTERNATIVE_NAME_MEMORY.equals(
                resourceName)) {
          if (memorySet) {
            throw new IllegalArgumentException(
                "Only one of the following keys " +
                    "can be specified for a single job: " +
                    MRJobConfig.RESOURCE_TYPE_ALTERNATIVE_NAME_MEMORY + ", " +
                    MRJobConfig.RESOURCE_TYPE_NAME_MEMORY);
          }
          String units = isEmpty(resourceRequest.getUnits()) ?
              ResourceUtils.getDefaultUnit(ResourceInformation.MEMORY_URI) :
                resourceRequest.getUnits();
          this.resourceCapability.setMemorySize(
              UnitsConversionUtil.convert(units, "Mi",
                  resourceRequest.getValue()));
          memorySet = true;
          String memoryKey = getMemoryKey(taskType);
          if (memoryKey != null && conf.get(memoryKey) != null) {
            LOG.warn("Configuration " + resourceTypePrefix + resourceName +
                "=" + resourceRequest.getValue() + resourceRequest.getUnits() +
                " is overriding the " + memoryKey + "=" + conf.get(memoryKey) +
                " configuration");

View on GitHub (pinned to 2add963021)

Solutions

1. Grep the effective job config (job.xml in the job submission dir) for '^mapreduce.(map|reduce).resource.memory' and delete one of the two aliases — keep memory-mb
2. If a framework sets one alias for you, stop setting the other in your own defaults
3. For programmatic configs, assert the invariant before submission (see validation snippet)

Example fix

<!-- before -->
<property><name>mapreduce.map.resource.memory</name><value>2048</value></property>
<property><name>mapreduce.map.resource.memory-mb</name><value>2048</value></property>

<!-- after - exactly one memory alias per task type -->
<property><name>mapreduce.map.resource.memory-mb</name><value>2048</value></property>

// fail fast at submit time instead of in the AM
for (String prefix : new String[]{"mapreduce.map.resource.", "mapreduce.reduce.resource."}) {
  boolean hasMemory = conf.get(prefix + "memory") != null;
  boolean hasMemoryMb = conf.get(prefix + "memory-mb") != null;
  if (hasMemory && hasMemoryMb) {
    throw new IllegalArgumentException(
        "Both " + prefix + "memory and " + prefix + "memory-mb set; remove one");
  }
}

When it happens

Trigger: Job config carries <prefix>memory and <prefix>memory-mb simultaneously (XML, -D on the command line twice, or programmatic conf.set of both); a template merges legacy and new-style resource keys; getRequestedResourcesFromConfig returns both aliases for the prefix.

Common situations: Upgrading jobs from old memory settings while scripts still append the legacy alias; config-management (Ansible/Amber) templates layering resource keys; copy-pasted examples mixing mapreduce.map.resource.memory-mb with mapreduce.map.resource.memory.

Related errors

• Only one of the following keys can be specified for a single
• Failed to parse value %s as %s, property key %s
• maxRetries = ${maxRetries} < 0
• sleepTime = ${sleepTime} < 0
• numRetries = ${numRetries} < 0

AI-assisted analysis of apache/hadoop@2add963021 (2026-08-22). Data as JSON: /api/errors/e43f2d6eecab8148. Report an issue: GitHub.