{"record":{"id":"cdc51f51d5757bb0","repo":"apache/hadoop","slug":"cannot-finalize-file-because-it-is-not-under-co","errorCode":null,"errorMessage":"Cannot finalize file {} because it is not under construction","messagePattern":"Cannot finalize file (.+?) because it is not under construction","errorType":"exception","errorClass":"IOException","httpStatus":null,"severity":"error","filePath":"hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/FSNamesystem.java","lineNumber":3985,"sourceCode":"    if (i < 0) {\n      i = 0;\n    }\n    for(; i < blocks.length; i++) {\n      final BlockInfo b = blocks[i];\n      if (b != null && b.getBlockUCState() == BlockUCState.COMMITTED) {\n        // b is COMMITTED but not yet COMPLETE, add it to pending replication.\n        blockManager.addExpectedReplicasToPending(b);\n      }\n    }\n  }\n\n  void finalizeINodeFileUnderConstruction(String src, INodeFile pendingFile,\n      int latestSnapshot, boolean allowCommittedBlock) throws IOException {\n    assert hasWriteLock(RwLockMode.GLOBAL);\n\n    FileUnderConstructionFeature uc = pendingFile.getFileUnderConstructionFeature();\n    if (uc == null) {\n      throw new IOException(\"Cannot finalize file \" + src\n          + \" because it is not under construction\");\n    }\n\n    pendingFile.recordModification(latestSnapshot);\n\n    // The file is no longer pending.\n    // Create permanent INode, update blocks. No need to replace the inode here\n    // since we just remove the uc feature from pendingFile\n    pendingFile.toCompleteFile(now(),\n        allowCommittedBlock? numCommittedAllowed: 0,\n        blockManager.getMinReplication());\n\n    leaseManager.removeLease(uc.getClientName(), pendingFile);\n\n    // close file and persist block allocations for this file\n    closeFile(src, pendingFile);\n\n    blockManager.checkRedundancy(pendingFile);","sourceCodeStart":3967,"sourceCodeEnd":4003,"githubUrl":"https://github.com/apache/hadoop/blob/2add9630210752f88ceb1bb74eb65e37bf41da8e/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/FSNamesystem.java#L3967-L4003","documentation":"finalizeINodeFileUnderConstruction throws IOException when the INodeFile no longer carries a FileUnderConstructionFeature, meaning it was already finalized or was never opened for writing. The method only converts open files to complete files, so a second finalize attempt is rejected as an invalid state transition.","triggerScenarios":"completeFile retried after a timeout when the first call already finalized the file; commitBlockSynchronization racing a lease recovery that closed the file first.","commonSituations":"Aggressive hand-rolled retry loops around completeFile; NameNode failover replaying an operation that had already succeeded; internal races fixed in later Hadoop versions.","solutions":["Before propagating the failure, re-check the file state: if it is closed with the expected length, treat the completion as successful (idempotent close)","Make client completeFile logic idempotent: on IOException, call getFileStatus and accept closed+full-length as success","Upgrade Hadoop if running an old 2.x line with known complete-vs-recovery races","Avoid immediate tight retries on completeFile; add a state check between attempts"],"exampleFix":"// before\nif (!dfsClient.complete(src, clientName)) { throw new IOException(\"complete failed\"); }\n\n// after\nif (!dfsClient.complete(src, clientName)) {\n  HdfsFileStatus st = dfsClient.getFileInfo(src);\n  if (st == null || !st.isClosed() || st.getLen() != expectedLen) {\n    throw new IOException(\"complete failed for \" + src);\n  } // else: already finalized by a retry, accept it\n}","handlingStrategy":"try-catch","validationCode":"HdfsFileStatus st = dfsClient.getFileInfo(src);\nif (st != null && st.isClosed() && st.getLen() == expectedLen) {\n  return; // already finalized; skip complete call\n}","typeGuard":null,"tryCatchPattern":"try {\n  dfsClient.complete(src, clientName);\n} catch (IOException e) {\n  HdfsFileStatus st = dfsClient.getFileInfo(src);\n  if (st == null || !st.isClosed()) { throw e; } // genuine failure\n  // else: double-finalize race, accept as success\n}","preventionTips":["Make completeFile idempotent: verify closed state before retrying","Space retries instead of tight loops racing lease recovery","Run a current Hadoop version with the complete/recovery races fixed"],"tags":["hdfs","namenode","double-completion","state-machine","idempotency"],"backgroundTag":"invalid-state-transition","analyzedSha":"2add9630210752f88ceb1bb74eb65e37bf41da8e","analyzedAt":"2026-08-22T19:55:07.957Z","schemaVersion":2},"datasetVersion":"2026-08-22T20:17:22.307Z"}