{"record":{"id":"39eb087398309f8c","repo":"pentaho/pentaho-kettle","slug":"vfs-provider-s3-transfer-close-error","errorCode":"vfs.provider.s3/transfer.close-error","errorMessage":"vfs.provider.s3/transfer.close-error","messagePattern":"vfs\\.provider\\.s3/transfer\\.close-error","errorType":"error_code","errorClass":"FileSystemException","httpStatus":null,"severity":"error","filePath":"plugins/s3-vfs/core/src/main/java/org/pentaho/s3common/S3TransferManager.java","lineNumber":99,"sourceCode":"      || dst.bucketName == null || dst.key == null ) {\n      throw new FileSystemException( \"vfs.provider.s3/transfer.null-argument\",\n        src != null ? src.getName().getURI() : \"null\",\n        dst != null ? dst.getQualifiedName() : \"null\" );\n    }\n    try ( InputStream in = src.getContent().getInputStream() ) {\n      String bucket = dst.bucketName;\n      String key = dst.key;\n      ObjectMetadata metadata = new ObjectMetadata();\n      metadata.setContentLength( src.getContent().getSize() );\n      TransferManager tm = getTransferManager();\n      tm.upload( bucket, key, in, metadata ).waitForUploadResult();\n    } catch ( InterruptedException e ) {\n      Thread.currentThread().interrupt();\n      throw new FileSystemException( \"vfs.provider.s3/transfer.interrupted\", e );\n    } catch ( AmazonClientException e ) {\n      throw new FileSystemException( \"vfs.provider.s3/transfer.error\", e );\n    } catch ( IOException e ) {\n      throw new FileSystemException( \"vfs.provider.s3/transfer.close-error\", e );\n    }\n  }\n}\n","sourceCodeStart":81,"sourceCodeEnd":103,"githubUrl":"https://github.com/pentaho/pentaho-kettle/blob/f3058517a153da500bf4551f46d79b91bf8ec552/plugins/s3-vfs/core/src/main/java/org/pentaho/s3common/S3TransferManager.java#L81-L103","documentation":"S3TransferManager.upload wraps the transfer lifecycle; if closing the underlying S3 object input/output stream throws an IOException after the upload itself, a FileSystemException with key 'vfs.provider.s3/transfer.close-error' is thrown. It signals cleanup failure, not a failed transfer of data.","triggerScenarios":"Commons VFS upload() on an S3 file object completes the copy but data.s3ObjectInputStream (or output stream) close() raises an IOException, e.g. stream already closed, broken connection while draining the stream, or an S3 client stream that validates content MD5/length on close.","commonSituations":"Network drop at end of transfer; aborting a transfer mid-stream; calling close twice on the VFS FileObject; S3 SDK stream internals failing on close after connection pool shutdown.","solutions":["Wrap the FileObject usage in try-with-resources so VFS close is called once and errors surface from the resource management, not mid-operation","Check network/proxy stability; ensure the stream is fully consumed or aborted before close","Upgrade the AWS SDK / pentaho-s3-vfs plugin — newer SDKs handle close on aborted transfers gracefully","Catch FileSystemException and inspect the cause; if the upload data itself succeeded, treat as cleanup-only and retry close"],"exampleFix":"// before\nFileObject src = fsManager.resolveFile( \"s3://bucket/key\" );\nsrc.copyTo( dest );\nsrc.close(); // IOException here surfaces as transfer.close-error\n// after\ntry ( FileObject src = fsManager.resolveFile( \"s3://bucket/key\" ) ) {\n  src.copyTo( dest );\n} // single close, exception from copy vs close distinguishable via cause","handlingStrategy":"try-catch","validationCode":"FileObject src = fsManager.resolveFile( \"s3://bucket/key\" );\nif ( !src.exists() || !src.getContent().isOpenable() ) throw new IllegalStateException( \"bad source\" );","typeGuard":null,"tryCatchPattern":"try ( FileObject src = fsManager.resolveFile( \"s3://bucket/key\" ) ) {\n  src.copyTo( dest );\n} catch ( FileSystemException e ) {\n  if ( \"vfs.provider.s3/transfer.close-error\".equals( e.getCode() ) && transferDataIntact ) {\n    log.warn( \"cleanup close failed; data transferred\", e );\n  } else {\n    throw e;\n  }\n}","preventionTips":["Always use try-with-resources or a single close path per FileObject","Do not close underlying S3 streams manually; let VFS manage them","Keep the AWS SDK and s3-vfs plugin updated","Ensure streams are fully consumed or aborted before close"],"tags":["s3","io","vfs","stream-close"],"backgroundTag":"file-close-failed","analyzedSha":"f3058517a153da500bf4551f46d79b91bf8ec552","analyzedAt":"2026-09-13T14:04:16.340Z","contentChangedAt":"2026-09-13T14:04:16.340Z","schemaVersion":2},"datasetVersion":"2026-09-20T23:17:15.980Z"}