{"record":{"id":"90e6d57a2760e3c8","repo":"apache/hadoop","slug":"failed-to-create-store-operations-from-configurati","errorCode":null,"errorMessage":"Failed to create Store Operations from configuration option mapreduce.manifest.committer.store.operations.classname:{e}","messagePattern":"Failed to create Store Operations from configuration option mapreduce\\.manifest\\.committer\\.store\\.operations\\.classname:(.+?)","errorType":"exception","errorClass":"PathIOException","httpStatus":null,"severity":"error","filePath":"hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/java/org/apache/hadoop/mapreduce/lib/output/committer/manifest/impl/ManifestCommitterSupport.java","lineNumber":290,"sourceCode":"   * @param path path under FS.\n   * @return a bonded store operations.\n   * @throws IOException on binding/init problems.\n   */\n  public static ManifestStoreOperations createManifestStoreOperations(\n      final Configuration conf,\n      final FileSystem filesystem,\n      final Path path) throws IOException {\n    try {\n      final Class<? extends ManifestStoreOperations> storeClass = conf.getClass(\n          OPT_STORE_OPERATIONS_CLASS,\n          ManifestStoreOperationsThroughFileSystem.class,\n          ManifestStoreOperations.class);\n      final ManifestStoreOperations operations = storeClass.\n          getDeclaredConstructor().newInstance();\n      operations.bindToFileSystem(filesystem, path);\n      return operations;\n    } catch (Exception e) {\n      throw new PathIOException(path.toString(),\n          \"Failed to create Store Operations from configuration option \"\n              + OPT_STORE_OPERATIONS_CLASS\n              + \":\" + e, e);\n    }\n  }\n\n  /**\n   * Logic to create directory names from job and attempt.\n   * This is self-contained it so it can be used in tests\n   * as well as in the committer.\n   */\n  public static class AttemptDirectories {\n\n    /**\n     * Job output path.\n     */\n    private final Path outputPath;\n","sourceCodeStart":272,"sourceCodeEnd":308,"githubUrl":"https://github.com/apache/hadoop/blob/2add9630210752f88ceb1bb74eb65e37bf41da8e/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/java/org/apache/hadoop/mapreduce/lib/output/committer/manifest/impl/ManifestCommitterSupport.java#L272-L308","documentation":"The manifest committer creates its store-operations layer from mapreduce.manifest.committer.store.operations.classname (default ManifestStoreOperationsThroughFileSystem): it loads the class, calls its no-arg constructor, then bindToFileSystem(fs, path). Any failure in that chain is wrapped in PathIOException('Failed to create Store Operations from configuration option ...:<cause>'). The nested exception is the actual story - wrong type, missing constructor, or bind-time validation.","triggerScenarios":"Configuring a custom ManifestStoreOperations class that (1) has no public no-arg constructor, (2) does not implement/extend ManifestStoreOperations (conf.getClass with that target type fails), or (3) whose bindToFileSystem() throws (unsupported filesystem, null filesystem, path validation).","commonSituations":"Plugging in a custom store-operations class for tuning or testing; class present at compile time but missing from the task-side jar; refactoring that adds constructor parameters; binding a store-operations implementation tied to a specific filesystem (e.g. ABFS) and then running against another FS.","solutions":["Unwrap the cause (PathIOException.getCause()): ClassNotFoundException/NoSuchMethodException = packaging/constructor problem; other exceptions = bindToFileSystem rejected the filesystem/path.","Ensure the class is public, implements ManifestStoreOperations, and declares a public no-arg constructor; move any setup into bindToFileSystem().","Ship the class in the job jar and confirm it loads on both client and task nodes (identical version).","If the bind failed, verify the filesystem/path combination is what your implementation supports (scheme, authority, permissions)."],"exampleFix":"// before: only a constructor with arguments\npublic class MyStoreOps implements ManifestStoreOperations {\n  public MyStoreOps(FileSystem fs) { }\n}\n\n// after: no-arg ctor + bind hook\npublic class MyStoreOps implements ManifestStoreOperations {\n  public MyStoreOps() { }\n  @Override public void bindToFileSystem(FileSystem fs, Path path) {\n    // store fs/path here\n  }\n}","handlingStrategy":"try-catch","validationCode":"// verify the configured store-operations class is loadable and constructible\nString cn = conf.get(\"mapreduce.manifest.committer.store.operations.classname\", \"\");\nif (!cn.isEmpty()) {\n  Class<?> c = Class.forName(cn);\n  Preconditions.checkState(ManifestStoreOperations.class.isAssignableFrom(c), \"wrong type: \" + cn);\n  c.getDeclaredConstructor().newInstance(); // fails fast if no public no-arg ctor\n}","typeGuard":null,"tryCatchPattern":"try {\n  ManifestStoreOperations ops = ManifestCommitterSupport.createStoreOperations(conf, fs, path);\n} catch (PathIOException e) {\n  Throwable cause = e.getCause() != null ? e.getCause() : e;\n  // distinguish packaging/ctor problems (fix the class) from bind failures (fix the fs/path)\n}","preventionTips":["Custom store-operations classes need a public no-arg constructor plus bindToFileSystem for setup.","Ship custom store operations in the job jar; verify the class resolves on task nodes too.","Smoke-test createStoreOperations in a unit test with the real filesystem/path shapes you use."],"tags":["hadoop","manifest-committer","reflection","configuration","classpath"],"backgroundTag":"class-instantiation-failed","analyzedSha":"2add9630210752f88ceb1bb74eb65e37bf41da8e","analyzedAt":"2026-08-22T19:55:07.957Z","schemaVersion":2},"datasetVersion":"2026-08-22T20:17:22.307Z"}