pentaho/pentaho-kettle · error · FileSystemException

vfs.provider.s3/transfer.close-error

vfs.provider.s3/transfer.close-error

Error message

vfs.provider.s3/transfer.close-error

What it means

S3TransferManager.upload wraps the transfer lifecycle; if closing the underlying S3 object input/output stream throws an IOException after the upload itself, a FileSystemException with key 'vfs.provider.s3/transfer.close-error' is thrown. It signals cleanup failure, not a failed transfer of data.

Solutions

1. Wrap the FileObject usage in try-with-resources so VFS close is called once and errors surface from the resource management, not mid-operation
2. Check network/proxy stability; ensure the stream is fully consumed or aborted before close
3. Upgrade the AWS SDK / pentaho-s3-vfs plugin — newer SDKs handle close on aborted transfers gracefully
4. Catch FileSystemException and inspect the cause; if the upload data itself succeeded, treat as cleanup-only and retry close

Example fix

// before
FileObject src = fsManager.resolveFile( "s3://bucket/key" );
src.copyTo( dest );
src.close(); // IOException here surfaces as transfer.close-error
// after
try ( FileObject src = fsManager.resolveFile( "s3://bucket/key" ) ) {
  src.copyTo( dest );
} // single close, exception from copy vs close distinguishable via cause

FileObject src = fsManager.resolveFile( "s3://bucket/key" );
if ( !src.exists() || !src.getContent().isOpenable() ) throw new IllegalStateException( "bad source" );

try ( FileObject src = fsManager.resolveFile( "s3://bucket/key" ) ) {
  src.copyTo( dest );
} catch ( FileSystemException e ) {
  if ( "vfs.provider.s3/transfer.close-error".equals( e.getCode() ) && transferDataIntact ) {
    log.warn( "cleanup close failed; data transferred", e );
  } else {
    throw e;
  }
}

When it happens

Trigger: Commons VFS upload() on an S3 file object completes the copy but data.s3ObjectInputStream (or output stream) close() raises an IOException, e.g. stream already closed, broken connection while draining the stream, or an S3 client stream that validates content MD5/length on close.

Common situations: Network drop at end of transfer; aborting a transfer mid-stream; calling close twice on the VFS FileObject; S3 SDK stream internals failing on close after connection pool shutdown.

Related errors

• Unable to close file channel for file '
• could not connect to pipedInputStream
• Error opening new file
• Exception getting the list of buckets
• Exception reading line using NIO

AI-assisted analysis of pentaho/pentaho-kettle@f3058517a1 (2026-09-13). Data as JSON: /api/errors/39eb087398309f8c. Report an issue: GitHub.

Appendix: source

Thrown at plugins/s3-vfs/core/src/main/java/org/pentaho/s3common/S3TransferManager.java:99

      || dst.bucketName == null || dst.key == null ) {
      throw new FileSystemException( "vfs.provider.s3/transfer.null-argument",
        src != null ? src.getName().getURI() : "null",
        dst != null ? dst.getQualifiedName() : "null" );
    }
    try ( InputStream in = src.getContent().getInputStream() ) {
      String bucket = dst.bucketName;
      String key = dst.key;
      ObjectMetadata metadata = new ObjectMetadata();
      metadata.setContentLength( src.getContent().getSize() );
      TransferManager tm = getTransferManager();
      tm.upload( bucket, key, in, metadata ).waitForUploadResult();
    } catch ( InterruptedException e ) {
      Thread.currentThread().interrupt();
      throw new FileSystemException( "vfs.provider.s3/transfer.interrupted", e );
    } catch ( AmazonClientException e ) {
      throw new FileSystemException( "vfs.provider.s3/transfer.error", e );
    } catch ( IOException e ) {
      throw new FileSystemException( "vfs.provider.s3/transfer.close-error", e );
    }
  }
}

View on GitHub (pinned to f3058517a1)