apache/iceberg · error

Unable to close the manifest writer: %s

Error message

Unable to close the manifest writer: %s

What it means

In SparkTableUtil.buildManifest, after writing data file entries into a ManifestWriter inside try-with-resources, an IOException raised while closing the manifest writer is converted to this unchecked exception. It means the manifest metadata file for the import could not be finalized on the target filesystem, so imported files cannot be committed.

Source

Thrown at spark/v4.0/spark/src/main/java/org/apache/iceberg/spark/SparkTableUtil.java:236

      FileIO io = new HadoopFileIO(conf.get());
      TaskContext ctx = TaskContext.get();
      String suffix =
          String.format(
              Locale.ROOT,
              "stage-%d-task-%d-manifest-%s",
              ctx.stageId(),
              ctx.taskAttemptId(),
              UUID.randomUUID());
      Path location = new Path(basePath, suffix);
      String outputPath = FileFormat.AVRO.addExtension(location.toString());
      OutputFile outputFile = io.newOutputFile(outputPath);
      ManifestWriter<DataFile> writer =
          ManifestFiles.write(formatVersion, spec, outputFile, snapshotId);

      try (ManifestWriter<DataFile> writerRef = writer) {
        fileTuples.forEachRemaining(fileTuple -> writerRef.add(fileTuple._2));
      } catch (IOException e) {
        throw SparkExceptionUtil.toUncheckedException(
            e, "Unable to close the manifest writer: %s", outputPath);
      }

      ManifestFile manifestFile = writer.toManifestFile();
      return ImmutableList.of(manifestFile).iterator();
    } else {
      return Collections.emptyIterator();
    }
  }

  /**
   * Import files from an existing Spark table to an Iceberg table.
   *
   * <p>The import uses the Spark session to get table metadata. It assumes no operation is going on
   * the original and target table and thus is not thread-safe.
   *
   * @param spark a Spark session
   * @param sourceTableIdent an identifier of the source Spark table

View on GitHub (pinned to 86d9c8fc54)

Solutions

1. Check write permissions and available space/capacity on the Iceberg table's write location (write.location / warehouse path).
2. Verify storage credentials and connectivity (HDFS NameNode reachable, S3/GCS tokens valid) then retry the import.
3. Inspect the wrapped IOException cause (SparkExceptionUtil keeps the cause) to identify the underlying filesystem error.
4. Retry the import; failed manifests should not have been committed, so re-running is safe.

Example fix

// before
conf.set("fs.defaultFS", "hdfs://stale-namenode:8020");
// after
conf.set("fs.defaultFS", "hdfs://active-namenode:8020"); // ensure table location FS is reachable/writable before import

// Verify write access to the table location before import
FileIO io = targetTable.io();
io.newOutputFile(targetTable.location() + "/.write-check").create();

try { SparkTableUtil.importSparkTable(...); } catch (RuntimeException e) { Throwable c = e.getCause(); if (c instanceof IOException) { /* inspect filesystem error, retry */ } else throw e; }

When it happens

Trigger: Running SparkTableUtil.importSparkTable / importUnpartitionedSparkTable where closing the OutputFile (via ManifestWriter.close) fails with IOException, e.g. the outputPath cannot be flushed to storage.

Common situations: HDFS/S3 outages or permission errors on the Iceberg table location; disk full on local warehouse; credentials expiring mid-write for cloud storage; network interruption to the object store during close.

Understand the failure class

Background: "failed to write file", "Could not save figure", "Error saving remote file" — file write failed: causes and fixes across languages and libraries — this error's family across 38 libraries.

Related errors

• Failed to close current writer
• Failed to close manifest reader
• Failed to close entries while caching changes
• Failed to create snapshot list writer for path: %s
• Cannot read manifest list file: %s

AI-assisted analysis of apache/iceberg@86d9c8fc54 (2026-09-12). Data as JSON: /api/errors/985ab93102acb157. Report an issue: GitHub.