pentaho/pentaho-kettle · error · FileSystemException

vfs.provider.s3/transfer.error

vfs.provider.s3/transfer.error

Error message

vfs.provider.s3/transfer.error

What it means

Thrown by S3TransferManager.copy() when the AWS TransferManager copy fails with an AmazonClientException (the S3->S3 server-side copy did not succeed). It wraps the underlying AWS SDK failure, e.g. access denied, missing source object, throttling, or network problems.

Solutions

1. Read the wrapped AmazonClientException/AmazonS3Exception message for the precise S3 error code
2. Verify IAM permissions for both source (GetObject) and destination (PutObject) buckets
3. Confirm the source object still exists and buckets are compatible (region, encryption)
4. Enable retry/timeout tuning on the AWS client and retry transient errors with backoff

Example fix

// before
} catch ( AmazonClientException e ) {
  throw new FileSystemException( "vfs.provider.s3/transfer.error", src.getQualifiedName(), dst.getQualifiedName(), e );
}
// after
} catch ( AmazonClientException e ) {
  if ( e instanceof AmazonS3Exception && "NoSuchKey".equals( ((AmazonS3Exception) e).getErrorCode() ) ) {
    logger.warn( "Source vanished: {}", src.getQualifiedName() );
  }
  throw new FileSystemException( "vfs.provider.s3/transfer.error", src.getQualifiedName(), dst.getQualifiedName(), e );
}

// preflight both sides
s3Client.doesObjectExist( src.bucketName, src.key );
s3Client.getBucketLocation( dst.bucketName ); // throws if bucket/creds/region bad

boolean canServerSideCopy( S3CommonFileObject src, S3CommonFileObject dst ) {
  return isResolvedS3File( src ) && isResolvedS3File( dst )
    && s3Client.doesObjectExist( src.bucketName, src.key );
}

try {
  transferManager.copy( src, dst );
} catch ( FileSystemException e ) {
  Throwable cause = e.getCause();
  if ( cause instanceof AmazonS3Exception && "AccessDenied".equals( ((AmazonS3Exception) cause).getErrorCode() ) ) {
    // fix IAM: GetObject on src, PutObject on dst
  }
  throw e;
}

When it happens

Trigger: Calling copy() when S3 rejects or fails the server-side CopyObject: source object missing, IAM lacking s3:GetObject on source or s3:PutObject on destination, SSE/KMS mismatch, or client/network error.

Common situations: Cross-account or cross-region copies without proper permissions; source deleted concurrently; KMS-encrypted buckets with mismatched key policies; offline/timeout network conditions.

Related errors

• FileSystemException wrapping SdkClientException
• vfs.provider/copy-file.error
• vfs.provider/copy-missing-file.error
• vfs.provider/get-type.error
• vfs.provider/get-type.error

AI-assisted analysis of pentaho/pentaho-kettle@f3058517a1 (2026-09-13). Data as JSON: /api/errors/27e8db2587cf6dc9. Report an issue: GitHub.

Appendix: source

Thrown at plugins/s3-vfs/core/src/main/java/org/pentaho/s3common/S3TransferManager.java:74

      || dst.bucketName == null || dst.key == null ) {
      throw new FileSystemException( "vfs.provider.s3/transfer.null-argument",
        src != null ? src.getQualifiedName() : "null",
        dst != null ? dst.getQualifiedName() : "null" );
    }
    try {
      Copy copy = getTransferManager().copy(
        src.bucketName, src.key,
        dst.bucketName, dst.key
      );
      copy.waitForCompletion();
      logger.info( "S3->S3 server-side copy succeeded: {} -> {}",
                   src.getQualifiedName(), dst.getQualifiedName() );
    } catch ( InterruptedException ie ) {
      Thread.currentThread().interrupt();
      throw new FileSystemException( "vfs.provider.s3/transfer.interrupted",
                                     src.getQualifiedName(), dst.getQualifiedName(), ie );
    } catch ( AmazonClientException e ) {
      throw new FileSystemException( "vfs.provider.s3/transfer.error",
                                     src.getQualifiedName(), dst.getQualifiedName(), e );
    }
  }

  public void upload( FileObject src, S3CommonFileObject dst ) throws FileSystemException {
    if ( src == null || dst == null
      || dst.bucketName == null || dst.key == null ) {
      throw new FileSystemException( "vfs.provider.s3/transfer.null-argument",
        src != null ? src.getName().getURI() : "null",
        dst != null ? dst.getQualifiedName() : "null" );
    }
    try ( InputStream in = src.getContent().getInputStream() ) {
      String bucket = dst.bucketName;
      String key = dst.key;
      ObjectMetadata metadata = new ObjectMetadata();
      metadata.setContentLength( src.getContent().getSize() );
      TransferManager tm = getTransferManager();
      tm.upload( bucket, key, in, metadata ).waitForUploadResult();

View on GitHub (pinned to f3058517a1)