{"record":{"id":"304c9f86a307d602","repo":"apache/hadoop","slug":"e-no-magic-path-element","errorCode":"E_NO_MAGIC_PATH_ELEMENT","errorMessage":"No __magic_job- element in path","messagePattern":"No __magic_job- element in path","errorType":"exception","errorClass":"IllegalArgumentException","httpStatus":null,"severity":"error","filePath":"hadoop-tools/hadoop-aws/src/main/java/org/apache/hadoop/fs/s3a/commit/MagicCommitPaths.java","lineNumber":110,"sourceCode":"    return elements.contains(BASE);\n  }\n\n  /**\n   * Get the index of the magic path element.\n   * @param elements full path element list\n   * @return the index.\n   * @throws IllegalArgumentException if there is no magic element\n   */\n  public static int magicElementIndex(List<String> elements) {\n    Optional<Integer> index = IntStream.range(0, elements.size())\n        .filter(i -> elements.get(i).startsWith(MAGIC_PATH_PREFIX))\n        .boxed()\n        .findFirst();\n\n    if (index.isPresent()) {\n      return index.get();\n    } else {\n      throw new IllegalArgumentException(E_NO_MAGIC_PATH_ELEMENT);\n    }\n  }\n\n  /**\n   * Get the parent path elements of the magic path.\n   * The list may be immutable or may be a view of the underlying list.\n   * Both the parameter list and the returned list MUST NOT be modified.\n   * @param elements full path element list\n   * @return the parent elements; may be empty\n   */\n  public static List<String> magicPathParents(List<String> elements) {\n    return elements.subList(0, magicElementIndex(elements));\n  }\n\n  /**\n   * Get the child path elements under the magic path.\n   * The list may be immutable or may be a view of the underlying list.\n   * Both the parameter list and the returned list MUST NOT be modified.","sourceCodeStart":92,"sourceCodeEnd":128,"githubUrl":"https://github.com/apache/hadoop/blob/2add9630210752f88ceb1bb74eb65e37bf41da8e/hadoop-tools/hadoop-aws/src/main/java/org/apache/hadoop/fs/s3a/commit/MagicCommitPaths.java#L92-L128","documentation":"Unchecked IllegalArgumentException from MagicCommitPaths.magicElementIndex when no element of the supplied path element list starts with '__magic_job-'. This utility backs the magic-committer path arithmetic (finding the magic element, splitting parents/children, extracting the job id) and assumes the caller already knows the path is a magic path. When the magic element is absent there is no meaningful index to return, so it throws rather than returning -1 or Optional.","triggerScenarios":"Calling magicElementIndex or the methods built on it (magicPathParents, magicPathChildren, pathToJobId) with the element list of a path like s3a://bucket/output/part-00000 that contains no __magic_job-* element.","commonSituations":"Calling MagicCommitPaths helpers on arbitrary user paths without first checking isMagicCommitPath; refactoring that routes non-magic paths into magic-path code; paths constructed from strings that lost the magic element during joining or normalization.","solutions":["Gate every call with S3AFileSystem.isMagicCommitPath(path) or CommitUtils.verifyIsMagicCommitPath before using MagicCommitPaths helpers","Scan the element list yourself for a component starting with '__magic_job-' and skip the magic handling when absent","Because this is an unchecked exception, wrap committer-adjacent code that touches paths in a boundary that converts it into a reported job error"],"exampleFix":"// before\nList<String> elements = MagicCommitPaths.pathToElements(path);\nint i = MagicCommitPaths.magicElementIndex(elements); // throws for non-magic paths\n\n// after\nif (!fs.isMagicCommitPath(path)) {\n  // regular path handling\n} else {\n  int i = MagicCommitPaths.magicElementIndex(MagicCommitPaths.pathToElements(path));\n}","handlingStrategy":"validation","validationCode":"List<String> elements = MagicCommitPaths.pathToElements(path);\nboolean hasMagic = elements.stream()\n    .anyMatch(e -> e.startsWith(CommitConstants.MAGIC_PATH_PREFIX));\nif (!hasMagic) {\n  // plain path -- take the non-magic code route\n}","typeGuard":null,"tryCatchPattern":"try {\n  int idx = MagicCommitPaths.magicElementIndex(elements);\n} catch (IllegalArgumentException e) {\n  // unchecked: guard at the API boundary so it cannot escape as a raw runtime error\n  throw new IOException(\"Path has no magic element: \" + elements, e);\n}","preventionTips":["Treat MagicCommitPaths helpers as internal-to-magic-paths only; never call them on arbitrary user paths","Wrap path-manipulation code that handles both magic and regular paths behind one isMagic check","Remember this is an unchecked exception -- static typing will not save you"],"tags":["s3a","magic-committer","path-validation","illegal-argument"],"backgroundTag":"path-validation-failed","analyzedSha":"2add9630210752f88ceb1bb74eb65e37bf41da8e","analyzedAt":"2026-08-22T19:55:07.957Z","schemaVersion":2},"datasetVersion":"2026-08-22T20:17:22.307Z"}