apache/beam · error · IOException

Failed closing channel to %s

Error message

Failed closing channel to %s

What it means

Writer.close()/finalize output wraps any exception from closing the underlying WritableByteChannel into an IOException 'Failed closing channel to <outputFile>'. This signals the temp output file's channel could not be cleanly closed before the file is committed.

Source

Thrown at sdks/java/core/src/main/java/org/apache/beam/sdk/io/FileBasedSink.java:1074

      } catch (Exception e) {
        closeChannelAndThrow(channel, outputFile, e);
      }

      try {
        finishWrite();
      } catch (Exception e) {
        closeChannelAndThrow(channel, outputFile, e);
      }

      // It is valid for a subclass to either close the channel or not.
      // They would typically close the channel e.g. if they are wrapping it in another channel
      // and the wrapper needs to be closed.
      if (channel.isOpen()) {
        LOG.debug("Closing channel to {}.", outputFile);
        try {
          channel.close();
        } catch (Exception e) {
          throw new IOException(String.format("Failed closing channel to %s", outputFile), e);
        }
      }
      LOG.info("Successfully wrote temporary file {}", outputFile);
    }

    /** Return the WriteOperation that this Writer belongs to. */
    public WriteOperation<DestinationT, OutputT> getWriteOperation() {
      return writeOperation;
    }

    void setDestination(DestinationT destination) {
      this.destination = destination;
    }

    /** Return the user destination object for this writer. */
    public DestinationT getDestination() {
      return destination;
    }

View on GitHub (pinned to 12126d8942)

Solutions

1. Retry the pipeline/job; transient storage outages often resolve.
2. Check storage backend connectivity, credentials, and quotas for the temp/output locations.
3. Verify the temporary and output directories are writable by the runner's service account.
4. Inspect the chained cause 'e' for the underlying I/O error.

try { writer.close(); } catch (IOException e) { if (isTransient(e.getCause())) { retryWithBackoff(); } else { throw e; } }

When it happens

Trigger: channel.close() throwing while finishing a sink write — underlying filesystem I/O failure, network filesystem (GCS/S3/HDFS) outage, or disk issues during WriteOperation finalization.

Common situations: Transient GCS/S3/HDFS connectivity problems during write finalize, disk full, or permissions changes mid-write in distributed runners.

Understand the failure class

Background: "failed to write file", "Could not save figure", "Error saving remote file" — file write failed: causes and fixes across languages and libraries — this error's family across 38 libraries.

Related errors

• Error determining if %s allows dynamic splitting
• File spec %s not found
• Error matching file spec %s: status %s
• Failed to get metadata from MatchResult: %s.
• Cannot estimate the row count. All the sampled lines are emp

AI-assisted analysis of apache/beam@12126d8942 (2026-09-13). Data as JSON: /api/errors/b9690e3193693855. Report an issue: GitHub.