{"record":{"id":"32b5fa0082331a20","repo":"apache/hadoop","slug":"incompatible-with-localrunner","errorCode":null,"errorMessage":"Incompatible with LocalRunner","messagePattern":"Incompatible with LocalRunner","errorType":"exception","errorClass":"UnsupportedOperationException","httpStatus":null,"severity":"error","filePath":"hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app/src/main/java/org/apache/hadoop/mapred/YarnOutputFiles.java","lineNumber":198,"sourceCode":"   * @return path\n   * @throws IOException\n   */\n  public Path getSpillIndexFileForWrite(int spillNumber, long size)\n      throws IOException {\n    return lDirAlloc.getLocalPathForWrite(\n        String.format(SPILL_INDEX_FILE_PATTERN,\n            conf.get(JobContext.TASK_ATTEMPT_ID), spillNumber), size, conf);\n  }\n\n  /**\n   * Return a local reduce input file created earlier\n   * \n   * @param mapId a map task id\n   * @return path\n   * @throws IOException \n   */\n  public Path getInputFile(int mapId) throws IOException {\n    throw new UnsupportedOperationException(\"Incompatible with LocalRunner\");\n  }\n\n  /**\n   * Create a local reduce input file name.\n   * \n   * @param mapId a map task id\n   * @param size the size of the file\n   * @return path\n   * @throws IOException\n   */\n  public Path getInputFileForWrite(org.apache.hadoop.mapreduce.TaskID mapId,\n      long size) throws IOException {\n    return lDirAlloc.getLocalPathForWrite(String.format(\n        REDUCE_INPUT_FILE_FORMAT_STRING,\n        getAttemptOutputDir().toString(), mapId.getId()),\n        size, conf);\n  }\n","sourceCodeStart":180,"sourceCodeEnd":216,"githubUrl":"https://github.com/apache/hadoop/blob/2add9630210752f88ceb1bb74eb65e37bf41da8e/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app/src/main/java/org/apache/hadoop/mapred/YarnOutputFiles.java#L180-L216","documentation":"YarnOutputFiles is the MapOutputFile implementation that YarnChild installs for tasks running on YARN (via MRConfig.TASK_LOCAL_OUTPUT_CLASS). On YARN, reduce inputs arrive over shuffle HTTP, so getLocalMapOutput-style handoff is meaningless; getInputFile(int mapId) exists only for the LocalJobRunner's in-JVM shuffle and always throws UnsupportedOperationException('Incompatible with LocalRunner'). Hitting it means local-runner code path (e.g. ReduceTask's local branch at ReduceTask.java:198, or a test at TestMapRed.java:310) executed against the YARN implementation -- a wiring or framework mismatch, never a runtime IO condition.","triggerScenarios":"Running with mapreduce.framework.name=local (LocalJobRunner) while MRConfig.TASK_LOCAL_OUTPUT_CLASS is pinned to YarnOutputFiles; unit tests instantiating YarnOutputFiles directly and calling getInputFile; mixed-version jars where the local runner's own output-files class is absent so YarnOutputFiles resolves.","commonSituations":"Test suites moved from MiniMRCluster to LocalJobRunner with stale configuration; jobs that hardcode task output classes; downstream code copying Hadoop's internal local-shuffle path.","solutions":["Let each framework wire its own implementation: framework=local uses the local runner's MapOutputFile, framework=yarn uses YarnOutputFiles -- remove any manual setClass(MRConfig.TASK_LOCAL_OUTPUT_CLASS, ...)","In code, branch on instanceof before calling local-only methods, or select the implementation from JobConf: conf.getClass(MRConfig.TASK_LOCAL_OUTPUT_CLASS, ...)","Align hadoop-mapreduce-client-* jar versions on the classpath so LocalJobRunner finds its own output-files class"],"exampleFix":"// before: local-runner code path calling the YARN implementation\nPath in = mapOutputFile.getInputFile(mapId); // throws under YarnOutputFiles\n// after: guard by implementation before the local-shuffle handoff\nif (mapOutputFile instanceof YarnOutputFiles) {\n  throw new IllegalStateException(\"local shuffle handoff not available on YARN\");\n}\nPath in = mapOutputFile.getInputFile(mapId);","handlingStrategy":"type-guard","validationCode":null,"typeGuard":"// narrow before calling local-runner-only MapOutputFile methods\nprivate static boolean supportsLocalShuffleHandoff(MapOutputFile mof) {\n  return !(mof instanceof org.apache.hadoop.mapred.YarnOutputFiles);\n}\n\n// usage\nif (supportsLocalShuffleHandoff(mapOutputFile)) {\n  Path reduceIn = mapOutputFile.getInputFile(mapId);\n}","tryCatchPattern":"Catch UnsupportedOperationException around getInputFile and translate it into a configuration error ('local shuffle handoff used with YARN task output class') instead of letting it kill the task.","preventionTips":["Never hardcode MRConfig.TASK_LOCAL_OUTPUT_CLASS; let YarnChild/LocalJobRunner wire their own","Keep mapreduce.framework.name consistent with the runtime actually in use","In unit tests, instantiate the local runner's output-files implementation explicitly"],"tags":["mapreduce","yarn","local-runner","framework-mismatch","unsupported-operation","shuffle"],"backgroundTag":"unsupported-operation-exception","analyzedSha":"2add9630210752f88ceb1bb74eb65e37bf41da8e","analyzedAt":"2026-08-22T19:55:07.957Z","schemaVersion":2},"datasetVersion":"2026-08-22T20:17:22.307Z"}