{"record":{"id":"985ab93102acb157","repo":"apache/iceberg","slug":"unable-to-close-the-manifest-writer-s-985ab9","errorCode":null,"errorMessage":"Unable to close the manifest writer: %s","messagePattern":"Unable to close the manifest writer: (.+?)","errorType":"exception","errorClass":null,"httpStatus":null,"severity":"error","filePath":"spark/v4.0/spark/src/main/java/org/apache/iceberg/spark/SparkTableUtil.java","lineNumber":236,"sourceCode":"      FileIO io = new HadoopFileIO(conf.get());\n      TaskContext ctx = TaskContext.get();\n      String suffix =\n          String.format(\n              Locale.ROOT,\n              \"stage-%d-task-%d-manifest-%s\",\n              ctx.stageId(),\n              ctx.taskAttemptId(),\n              UUID.randomUUID());\n      Path location = new Path(basePath, suffix);\n      String outputPath = FileFormat.AVRO.addExtension(location.toString());\n      OutputFile outputFile = io.newOutputFile(outputPath);\n      ManifestWriter<DataFile> writer =\n          ManifestFiles.write(formatVersion, spec, outputFile, snapshotId);\n\n      try (ManifestWriter<DataFile> writerRef = writer) {\n        fileTuples.forEachRemaining(fileTuple -> writerRef.add(fileTuple._2));\n      } catch (IOException e) {\n        throw SparkExceptionUtil.toUncheckedException(\n            e, \"Unable to close the manifest writer: %s\", outputPath);\n      }\n\n      ManifestFile manifestFile = writer.toManifestFile();\n      return ImmutableList.of(manifestFile).iterator();\n    } else {\n      return Collections.emptyIterator();\n    }\n  }\n\n  /**\n   * Import files from an existing Spark table to an Iceberg table.\n   *\n   * <p>The import uses the Spark session to get table metadata. It assumes no operation is going on\n   * the original and target table and thus is not thread-safe.\n   *\n   * @param spark a Spark session\n   * @param sourceTableIdent an identifier of the source Spark table","sourceCodeStart":218,"sourceCodeEnd":254,"githubUrl":"https://github.com/apache/iceberg/blob/86d9c8fc543e7c56c9f624eb725f76c9baff9570/spark/v4.0/spark/src/main/java/org/apache/iceberg/spark/SparkTableUtil.java#L218-L254","documentation":"In SparkTableUtil.buildManifest, after writing data file entries into a ManifestWriter inside try-with-resources, an IOException raised while closing the manifest writer is converted to this unchecked exception. It means the manifest metadata file for the import could not be finalized on the target filesystem, so imported files cannot be committed.","triggerScenarios":"Running SparkTableUtil.importSparkTable / importUnpartitionedSparkTable where closing the OutputFile (via ManifestWriter.close) fails with IOException, e.g. the outputPath cannot be flushed to storage.","commonSituations":"HDFS/S3 outages or permission errors on the Iceberg table location; disk full on local warehouse; credentials expiring mid-write for cloud storage; network interruption to the object store during close.","solutions":["Check write permissions and available space/capacity on the Iceberg table's write location (write.location / warehouse path).","Verify storage credentials and connectivity (HDFS NameNode reachable, S3/GCS tokens valid) then retry the import.","Inspect the wrapped IOException cause (SparkExceptionUtil keeps the cause) to identify the underlying filesystem error.","Retry the import; failed manifests should not have been committed, so re-running is safe."],"exampleFix":"// before\nconf.set(\"fs.defaultFS\", \"hdfs://stale-namenode:8020\");\n// after\nconf.set(\"fs.defaultFS\", \"hdfs://active-namenode:8020\"); // ensure table location FS is reachable/writable before import","handlingStrategy":"try-catch","validationCode":"// Verify write access to the table location before import\nFileIO io = targetTable.io();\nio.newOutputFile(targetTable.location() + \"/.write-check\").create();","typeGuard":null,"tryCatchPattern":"try { SparkTableUtil.importSparkTable(...); } catch (RuntimeException e) { Throwable c = e.getCause(); if (c instanceof IOException) { /* inspect filesystem error, retry */ } else throw e; }","preventionTips":["Ensure the table write location is writable and has free space","Use fresh credentials with long enough validity for the whole import","Retry imports; no commit happens if the manifest close fails"],"tags":["spark","io","manifest-writer","filesystem","unchecked-exception"],"backgroundTag":"file-write-failed","analyzedSha":"86d9c8fc543e7c56c9f624eb725f76c9baff9570","analyzedAt":"2026-09-12T00:46:39.097Z","contentChangedAt":"2026-09-12T00:46:39.097Z","schemaVersion":2},"datasetVersion":"2026-09-14T16:17:12.679Z"}