{"record":{"id":"edeea402a06934d6","repo":"apache/hadoop","slug":"resilient-commit-not-supported","errorCode":null,"errorMessage":"Resilient commit not supported","messagePattern":"Resilient commit not supported","errorType":"exception","errorClass":"UnsupportedOperationException","httpStatus":null,"severity":"error","filePath":"hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/java/org/apache/hadoop/mapreduce/lib/output/committer/manifest/impl/ManifestStoreOperations.java","lineNumber":266,"sourceCode":"   * Commit one file through any resilient API.\n   * This operation MUST rename source to destination,\n   * else raise an exception.\n   * The result indicates whether or not some\n   * form of recovery took place.\n   *\n   * If etags were collected during task commit, these will be\n   * in the entries passed in here.\n   *\n   * The base implementation always raises\n   * {@code UnsupportedOperationException}\n   * @param entry entry to commit\n   * @return the result of the commit\n   * @throws IOException failure.\n   * @throws UnsupportedOperationException if not available.\n   *\n   */\n  public CommitFileResult commitFile(FileEntry entry) throws IOException {\n    throw new UnsupportedOperationException(\"Resilient commit not supported\");\n  }\n\n  /**\n   * Outcome from the operation {@link #commitFile(FileEntry)}.\n   * As a rename failure MUST raise an exception, this result\n   * only declares whether or not some form of recovery took place.\n   */\n  public static final class CommitFileResult {\n\n    /** Did recovery take place? */\n    private final boolean recovered;\n\n    /** Time waiting for IO capacity, may be null. */\n    @Nullable\n    private final Duration waitTime;\n\n    /**\n     * Full commit result.","sourceCodeStart":248,"sourceCodeEnd":284,"githubUrl":"https://github.com/apache/hadoop/blob/2add9630210752f88ceb1bb74eb65e37bf41da8e/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/java/org/apache/hadoop/mapreduce/lib/output/committer/manifest/impl/ManifestStoreOperations.java#L248-L284","documentation":"ManifestStoreOperations.commitFile() is the optional 'resilient commit' hook (commit a file through a store-native API instead of rename). The base implementation always throws UnsupportedOperationException('Resilient commit not supported'). The hook is only invoked when storeSupportsResilientCommit() returns true - the base returns false, and the ABFS implementation overrides both - so this exception means a subclass advertised resilient support but did not actually implement commitFile().","triggerScenarios":"A custom ManifestStoreOperations (set via mapreduce.manifest.committer.store.operations.classname) whose storeSupportsResilientCommit() returns true without overriding commitFile(FileEntry); during file commit, AbstractJobOrTaskStage.commitFile() then routes through operations.commitFile(entry) and hits the base throw.","commonSituations":"Copy-pasting the ABFS store-operations flag into a custom class without porting its commitFile(); partial refactors where the capability flag survives but the implementation is dropped.","solutions":["If you do not implement a native commit API, make storeSupportsResilientCommit() return false so the committer uses the rename path.","If you need resilient commit, implement commitFile(FileEntry) to move source to dest or raise IOException - copy the pattern from the ABFS subclass of ManifestStoreOperationsThroughFileSystem.","Add a unit test asserting: storeSupportsResilientCommit()==true implies commitFile() completes on your store."],"exampleFix":"// before: capability declared, hook missing\npublic class MyStoreOps extends ManifestStoreOperations {\n  @Override public boolean storeSupportsResilientCommit() { return true; }\n}\n\n// after: keep the flag honest\npublic class MyStoreOps extends ManifestStoreOperations {\n  @Override public boolean storeSupportsResilientCommit() { return false; }\n  // OR implement: @Override public CommitFileResult commitFile(FileEntry e) { ... }\n}","handlingStrategy":"type-guard","validationCode":null,"typeGuard":"public static boolean resilientCommitIsImplemented(ManifestStoreOperations ops) {\n  if (!ops.storeSupportsResilientCommit()) {\n    return false; // base rename path will be used, safe\n  }\n  try {\n    ops.getClass().getDeclaredMethod(\"commitFile\", FileEntry.class);\n    return true; // subclass actually overrides the hook\n  } catch (NoSuchMethodException notOverridden) {\n    return false; // advertising support without commitFile() would throw UOE\n  }\n}","tryCatchPattern":"try {\n  CommitFileResult r = operations.commitFile(entry);\n} catch (UnsupportedOperationException e) {\n  // subclass contract violation: stop advertising resilient support or implement commitFile\n}","preventionTips":["Keep storeSupportsResilientCommit() and commitFile() overrides paired in any subclass.","Copy the ABFS implementation wholesale if you need resilient commit; do not just flip the flag.","Add a contract test: capability flag true implies commitFile() does not throw UOE."],"tags":["hadoop","manifest-committer","unsupported-operation","subclassing","api-contract"],"backgroundTag":"unsupported-operation","analyzedSha":"2add9630210752f88ceb1bb74eb65e37bf41da8e","analyzedAt":"2026-08-22T19:55:07.957Z","schemaVersion":2},"datasetVersion":"2026-08-22T20:17:22.307Z"}