{"record":{"id":"5170ace946ea9505","repo":"apache/hadoop","slug":"invalid-value-for-mapreduce-reduce-shuffle-input-b","errorCode":null,"errorMessage":"Invalid value for mapreduce.reduce.shuffle.input.buffer.percent: {maxInMemCopyUse}","messagePattern":"Invalid value for mapreduce\\.reduce\\.shuffle\\.input\\.buffer\\.percent: (.+?)","errorType":"exception","errorClass":"IllegalArgumentException","httpStatus":null,"severity":"error","filePath":"hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/java/org/apache/hadoop/mapreduce/task/reduce/MergeManagerImpl.java","lineNumber":169,"sourceCode":"    \n    this.reporter = reporter;\n    this.codec = codec;\n    this.combinerClass = combinerClass;\n    this.combineCollector = combineCollector;\n    this.reduceCombineInputCounter = reduceCombineInputCounter;\n    this.spilledRecordsCounter = spilledRecordsCounter;\n    this.mergedMapOutputsCounter = mergedMapOutputsCounter;\n    this.mapOutputFile = mapOutputFile;\n    this.mapOutputFile.setConf(jobConf);\n    \n    this.localFS = localFS;\n    this.rfs = ((LocalFileSystem)localFS).getRaw();\n    \n    final float maxInMemCopyUse =\n      jobConf.getFloat(MRJobConfig.SHUFFLE_INPUT_BUFFER_PERCENT,\n          MRJobConfig.DEFAULT_SHUFFLE_INPUT_BUFFER_PERCENT);\n    if (maxInMemCopyUse > 1.0 || maxInMemCopyUse < 0.0) {\n      throw new IllegalArgumentException(\"Invalid value for \" +\n          MRJobConfig.SHUFFLE_INPUT_BUFFER_PERCENT + \": \" +\n          maxInMemCopyUse);\n    }\n\n    // Allow unit tests to fix Runtime memory\n    this.memoryLimit = (long)(jobConf.getLong(\n        MRJobConfig.REDUCE_MEMORY_TOTAL_BYTES,\n        Runtime.getRuntime().maxMemory()) * maxInMemCopyUse);\n\n    this.ioSortFactor = jobConf.getInt(MRJobConfig.IO_SORT_FACTOR,\n        MRJobConfig.DEFAULT_IO_SORT_FACTOR);\n\n    final float singleShuffleMemoryLimitPercent =\n        jobConf.getFloat(MRJobConfig.SHUFFLE_MEMORY_LIMIT_PERCENT,\n            DEFAULT_SHUFFLE_MEMORY_LIMIT_PERCENT);\n    if (singleShuffleMemoryLimitPercent < 0.0f\n        || singleShuffleMemoryLimitPercent > 1.0f) {\n      throw new IllegalArgumentException(\"Invalid value for \"","sourceCodeStart":151,"sourceCodeEnd":187,"githubUrl":"https://github.com/apache/hadoop/blob/2add9630210752f88ceb1bb74eb65e37bf41da8e/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/java/org/apache/hadoop/mapreduce/task/reduce/MergeManagerImpl.java#L151-L187","documentation":"MergeManagerImpl's constructor reads mapreduce.reduce.shuffle.input.buffer.percent (fraction of the reducer's memory budget usable for in-memory shuffle data, default 0.70) and requires it within [0.0, 1.0]; otherwise it throws IllegalArgumentException and the reduce task fails at startup. memoryLimit is later computed as maxMemory (or mapreduce.reduce.memory.total.bytes) times this fraction.","triggerScenarios":"The job conf contains a value like '1.5', '-0.1', '70' (percent instead of fraction), or '0.7f' for mapreduce.reduce.shuffle.input.buffer.percent.","commonSituations":"Operators pasting a percentage where a fraction is expected; templated config files substituting wrong values; copies from documentation of other shuffle settings that use bytes or percents.","solutions":["Set mapreduce.reduce.shuffle.input.buffer.percent to a plain float between 0.0 and 1.0 (default 0.70).","Remove the override to fall back to the default.","If more shuffle memory is the goal, raise the reducer heap / mapreduce.reduce.memory.total.bytes instead of pushing the fraction above 1."],"exampleFix":"<!-- before: percent instead of fraction -->\n<property><name>mapreduce.reduce.shuffle.input.buffer.percent</name><value>70</value></property>\n<!-- after: fraction of heap -->\n<property><name>mapreduce.reduce.shuffle.input.buffer.percent</name><value>0.70</value></property>","handlingStrategy":"validation","validationCode":"// Validate before job submission\nfloat p = conf.getFloat(\"mapreduce.reduce.shuffle.input.buffer.percent\", 0.70f);\nif (p < 0.0f || p > 1.0f) { throw new IllegalArgumentException(\"mapreduce.reduce.shuffle.input.buffer.percent must be a fraction in [0,1]: \" + p); }","typeGuard":null,"tryCatchPattern":null,"preventionTips":["Remember these settings are fractions (0-1), not percents.","Centralize shuffle-memory tuning in one reviewed template instead of ad-hoc job overrides."],"tags":["hadoop","mapreduce","shuffle","config","memory","validation"],"backgroundTag":"config-validation-failed","analyzedSha":"2add9630210752f88ceb1bb74eb65e37bf41da8e","analyzedAt":"2026-08-22T19:55:07.957Z","schemaVersion":2},"datasetVersion":"2026-08-22T20:17:22.307Z"}