{"record":{"id":"5a278fbd3938fc45","repo":"apache/hadoop","slug":"cannot-close-tfile-in-the-middle-of-key-value-inse","errorCode":null,"errorMessage":"Cannot close TFile in the middle of key-value insertion.","messagePattern":"Cannot close TFile in the middle of key-value insertion\\.","errorType":"exception","errorClass":"IllegalStateException","httpStatus":null,"severity":"error","filePath":"hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/io/file/tfile/TFile.java","lineNumber":323,"sourceCode":"    }\n\n    /**\n     * Close the Writer. Resources will be released regardless of the exceptions\n     * being thrown. Future close calls will have no effect.\n     * \n     * The underlying FSDataOutputStream is not closed.\n     */\n    @Override\n    public void close() throws IOException {\n      if ((state == State.CLOSED)) {\n        return;\n      }\n      try {\n        // First try the normal finish.\n        // Terminate upon the first Exception.\n        if (errorCount == 0) {\n          if (state != State.READY) {\n            throw new IllegalStateException(\n                \"Cannot close TFile in the middle of key-value insertion.\");\n          }\n\n          finishDataBlock(true);\n\n          // first, write out data:TFile.meta\n          BlockAppender outMeta =\n              writerBCF\n                  .prepareMetaBlock(TFileMeta.BLOCK_NAME, COMPRESSION_NONE);\n          try {\n            tfileMeta.write(outMeta);\n          } finally {\n            outMeta.close();\n          }\n\n          // second, write out data:TFile.index\n          BlockAppender outIndex =\n              writerBCF.prepareMetaBlock(TFileIndex.BLOCK_NAME);","sourceCodeStart":305,"sourceCodeEnd":341,"githubUrl":"https://github.com/apache/hadoop/blob/2add9630210752f88ceb1bb74eb65e37bf41da8e/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/io/file/tfile/TFile.java#L305-L341","documentation":"Thrown by TFile.Writer.close() when closing a writer whose state is not READY, meaning a key or value append stream obtained from prepareAppendKey()/prepareAppendValue() was never closed. TFile is a strict state machine; close() only accepts the writer in the READY state (no record in progress). The IllegalStateException fires only when errorCount == 0, i.e. when the previous streams appeared to complete normally but were left open.","triggerScenarios":"Calling writer.close() while a DataOutputStream returned by prepareAppendKey() or prepareAppendValue() is still open; abandoning a key/value stream mid-record and then closing the writer; forgetting the close() inside a loop that breaks early.","commonSituations":"Exception paths where the key stream is opened but the writer is closed in a finally block without closing the intermediate stream; refactoring that removes a stream close; control-flow bugs (break/continue/return) that skip the inner close before the outer close.","solutions":["Close every stream returned by prepareAppendKey()/prepareAppendValue() before calling writer.close(), ideally with try-with-resources or try/finally","Prefer the single-shot writer.append(key, value) API, which manages the state machine internally","If a record must be abandoned, be aware the writer is already inconsistent: close the inner stream (which may throw) and expect close() to surface the first error rather than the IllegalState one"],"exampleFix":"// before\nDataOutputStream dos = writer.prepareAppendKey(-1);\ndos.write(key);\nwriter.close(); // IllegalStateException: still in IN_KEY state\n\n// after\ntry (DataOutputStream dos = writer.prepareAppendKey(-1)) {\n  dos.write(key);\n}\nwriter.close();","handlingStrategy":"validation","validationCode":"// Ensure no append stream is open before closing the writer\n// Pattern: every prepare* stream lives in try-with-resources\ntry (DataOutputStream k = writer.prepareAppendKey(-1)) {\n  k.write(key);\n}\ntry (DataOutputStream v = writer.prepareAppendValue(-1)) {\n  v.write(value);\n}\nwriter.close(); // only called when writer is back in READY state","typeGuard":null,"tryCatchPattern":"catch (IllegalStateException e) {\n  // a stream was left open: close inner streams in finally, then close the writer;\n  // data after the abandoned record is not recoverable in this file\n}","preventionTips":["Prefer writer.append(key, value) which cannot leave the state machine mid-record","Never call writer.close() from a finally block while a prepared stream may still be open; nest the stream's close first","Keep one record's stream lifecycle inside a single method so no path skips the close"],"tags":["hadoop","tfile","state-machine","resource-leak","writer"],"backgroundTag":"illegal-state-transition","analyzedSha":"2add9630210752f88ceb1bb74eb65e37bf41da8e","analyzedAt":"2026-08-22T19:55:07.957Z","schemaVersion":2},"datasetVersion":"2026-08-22T20:17:22.307Z"}