{"record":{"id":"c9f45825b55fef20","repo":"apache/hadoop","slug":"resource-type-pattern-is-not-implemented-yet","errorCode":null,"errorMessage":"Resource type PATTERN is not implemented yet. {}","messagePattern":"Resource type PATTERN is not implemented yet\\. (.+?)","errorType":"exception","errorClass":"IllegalArgumentException","httpStatus":null,"severity":"error","filePath":"hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-common/src/main/java/org/apache/hadoop/mapred/LocalDistributedCacheManager.java","lineNumber":153,"sourceCode":"        try {\n          path = resourcesToPaths.get(resource).get();\n        } catch (InterruptedException e) {\n          throw new IOException(e);\n        } catch (ExecutionException e) {\n          throw new IOException(e);\n        }\n        String pathString = path.toUri().toString();\n        String link = entry.getKey();\n        String target = new File(path.toUri()).getPath();\n        symlink(workDir, target, link);\n        \n        if (resource.getType() == LocalResourceType.ARCHIVE) {\n          localArchives.add(pathString);\n        } else if (resource.getType() == LocalResourceType.FILE) {\n          localFiles.add(pathString);\n        } else if (resource.getType() == LocalResourceType.PATTERN) {\n          //PATTERN is not currently used in local mode\n          throw new IllegalArgumentException(\"Resource type PATTERN is not \" +\n          \t\t\"implemented yet. \" + resource.getResource());\n        }\n        Path resourcePath;\n        try {\n          resourcePath = resource.getResource().toPath();\n        } catch (URISyntaxException e) {\n          throw new IOException(e);\n        }\n        LOG.info(String.format(\"Localized %s as %s\", resourcePath, path));\n        String cp = resourcePath.toUri().getPath();\n        if (classpaths.keySet().contains(cp)) {\n          localClasspaths.add(path.toUri().getPath().toString());\n        }\n      }\n    } finally {\n      if (exec != null) {\n        exec.shutdown();\n      }","sourceCodeStart":135,"sourceCodeEnd":171,"githubUrl":"https://github.com/apache/hadoop/blob/2add9630210752f88ceb1bb74eb65e37bf41da8e/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-common/src/main/java/org/apache/hadoop/mapred/LocalDistributedCacheManager.java#L135-L171","documentation":"LocalDistributedCacheManager emulates YARN distributed-cache localization when a job runs through LocalJobRunner (mapreduce.framework.name=local). During setup it classifies each LocalResource as FILE or ARCHIVE; LocalResourceType.PATTERN has no local implementation, so it throws IllegalArgumentException and the local job fails during setup.","triggerScenarios":"A job whose resource map contains a LocalResource with LocalResourceType.PATTERN is run locally - typically code that builds LocalResources for a YARN deployment and is then reused in local unit tests or embedded local execution.","commonSituations":"Framework or application code that sets PATTERN for pattern-based classpath archives; tests switching between LocalJobRunner and MiniMRYarnCluster; running a YARN-style MR pipeline in local mode without adapting resources.","solutions":["Replace the PATTERN resource with LocalResourceType.ARCHIVE (or FILE) for local runs","Run the test on MiniMRYarnCluster instead of the pure local runner","Filter PATTERN resources out before submitting locally rather than letting setup fail"],"exampleFix":"// before\nLocalResource r = LocalResource.newInstance(url,\n    LocalResourceType.PATTERN, LocalResourceVisibility.APPLICATION,\n    10L, timestamp, \"classes/.*.jar\");\n\n// after: local runner supports FILE and ARCHIVE only\nLocalResource r = LocalResource.newInstance(url,\n    LocalResourceType.ARCHIVE, LocalResourceVisibility.APPLICATION,\n    10L, timestamp);","handlingStrategy":"validation","validationCode":"// Before running locally, scan the resource map for unsupported types\nboolean hasPattern = resources.values().stream()\n    .anyMatch(r -> r.getType() == LocalResourceType.PATTERN);\nif (hasPattern && \"local\".equals(conf.get(\"mapreduce.framework.name\"))) {\n  throw new IllegalStateException(\n      \"LocalJobRunner does not support LocalResourceType.PATTERN; \"\n    + \"convert to ARCHIVE/FILE or run on MiniMRYarnCluster\");\n}","typeGuard":"static boolean isLocalRunnerCompatible(Map<String, LocalResource> rs) {\n  return rs.values().stream().noneMatch(\n      r -> r.getType() == LocalResourceType.PATTERN);\n}","tryCatchPattern":"try {\n  job.submit();\n} catch (Exception e) {\n  if (e.getCause() instanceof IllegalArgumentException iae\n      && iae.getMessage().contains(\"PATTERN\")) {\n    // strip or convert PATTERN resources, then rerun locally or on a cluster\n  }\n  throw e;\n}","preventionTips":["Keep two resource profiles: full YARN resources and a local profile with FILE/ARCHIVE only","Run integration tests on MiniMRYarnCluster when PATTERN semantics matter","Assert resource types in test setup before choosing the local runner"],"tags":["mapreduce","distributed-cache","local-mode","localresource","local-job-runner"],"backgroundTag":"unsupported-operation","analyzedSha":"2add9630210752f88ceb1bb74eb65e37bf41da8e","analyzedAt":"2026-08-22T19:55:07.957Z","schemaVersion":2},"datasetVersion":"2026-08-23T01:17:44.959Z"}