{"record":{"id":"03eec5919eeeee31","repo":"apache/hadoop","slug":"invalid-k-argument-must-be-of-the-form-k-pos1","errorCode":null,"errorMessage":"Invalid -k argument. Must be of the form -k pos1,[pos2], where pos is of the form f[.c]nr","messagePattern":"Invalid -k argument\\. Must be of the form -k pos1,\\[pos2\\], where pos is of the form f\\[\\.c\\]nr","errorType":"validation","errorClass":"IllegalArgumentException","httpStatus":null,"severity":"error","filePath":"hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/java/org/apache/hadoop/mapreduce/lib/partition/KeyFieldHelper.java","lineNumber":263,"sourceCode":"          token = st.nextToken();\n          if (token.equals(\".\")) {\n            token = st.nextToken();\n            key.endChar = Integer.parseInt(token);\n            if (st.hasMoreTokens()) {\n              token = st.nextToken();\n            } else {\n              return key;\n            }\n          }\n          do {\n            if (token.equals(\"n\")) {\n              key.numeric = true;\n            }\n            else if (token.equals(\"r\")) {\n              key.reverse = true;\n            }\n            else { \n              throw new IllegalArgumentException(\"Invalid -k argument. \" +\n               \"Must be of the form -k pos1,[pos2], where pos is of the form \" +\n               \"f[.c]nr\");\n            }\n            if (st.hasMoreTokens()) {\n              token = st.nextToken();\n            } else {\n              break;\n            }\n          } while (true);\n        }\n        return key;\n      }\n      throw new IllegalArgumentException(\"Invalid -k argument. \" +\n          \"Must be of the form -k pos1,[pos2], where pos is of the form \" +\n          \"f[.c]nr\");\n    }\n    return key;\n  }","sourceCodeStart":245,"sourceCodeEnd":281,"githubUrl":"https://github.com/apache/hadoop/blob/2add9630210752f88ceb1bb74eb65e37bf41da8e/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/java/org/apache/hadoop/mapreduce/lib/partition/KeyFieldHelper.java#L245-L281","documentation":"KeyFieldHelper parses GNU-sort style -k key specifications for KeyFieldBasedComparator and Hadoop Streaming. In parseKey, once the optional end position (pos2, the part after the comma) has been read, only the modifier flags 'n' (numeric) and 'r' (reverse) may follow. This IllegalArgumentException fires when any other token trails the end position inside the do-while loop.","triggerScenarios":"KeyFieldHelper.parseOption (or JobConf/streaming forwarding -k arguments) receives a spec such as '-k 2,3x', '-k 4.2,5.3b', or '-k 1,2,' — a token after pos2 that is neither 'n' nor 'r'. GNU sort flags like 'b' (skip blanks), 'd', 'f', 'i', 'M', 'h' are not supported and land here.","commonSituations":"Streaming jobs or mapreduce.partition.keycomparator.options values copied verbatim from a Linux 'sort' command line; adding an unsupported flag after the end field; a stray trailing comma or typo after the end position.","solutions":["Remove the invalid trailing token so pos2 is of the form f[.c] followed only by n/r flags, e.g. '-k 2,3n'","Strip GNU sort flags KeyFieldHelper does not implement ('b','d','f','i','M','h') — only 'n' and 'r' exist","Validate the -k string with a small KeyFieldHelper.parseOption() unit test before submitting the job"],"exampleFix":"// before\nconf.set(\"mapreduce.partition.keycomparator.options\", \"-k 2,3nb\"); // 'b' unsupported after pos2\n// after\nconf.set(\"mapreduce.partition.keycomparator.options\", \"-k 2,3n\");","handlingStrategy":"validation","validationCode":"private static final Pattern K_SPEC =\n    Pattern.compile(\"^\\\\d+(\\\\.\\\\d+)?[nr]*(,\\\\d+(\\\\.\\\\d+)?[nr]*)?$\");\n\nstatic boolean isValidKeySpec(String spec) {\n  String arg = spec.startsWith(\"-k\") ? spec.substring(2) : spec;\n  return !arg.isEmpty() && K_SPEC.matcher(arg).matches();\n}","typeGuard":null,"tryCatchPattern":"try {\n  helper.parseOption(keyArg);\n} catch (IllegalArgumentException e) {\n  throw new IllegalArgumentException(\"Bad -k spec '\" + keyArg + \"': \" + e.getMessage(), e);\n}","preventionTips":["Allow only n/r modifiers after each position in generated -k strings","Run the spec through KeyFieldHelper in a unit test for every comparator option you ship","Do not copy sort(1) flag sets verbatim; whitelist the exact supported grammar"],"tags":["mapreduce","sorting","streaming","argument-validation","configuration"],"backgroundTag":"invalid-command-line-argument","analyzedSha":"2add9630210752f88ceb1bb74eb65e37bf41da8e","analyzedAt":"2026-08-22T19:55:07.957Z","schemaVersion":2},"datasetVersion":"2026-08-22T20:17:22.307Z"}