pentaho/pentaho-kettle · error · FileSystemException

vfs.provider/copy-missing-file.error

vfs.provider/copy-missing-file.error

Error message

vfs.provider/copy-missing-file.error

What it means

S3CommonFileObject.copyFrom() first validates that the source FileObject exists before walking it with the selector. If the source cannot be resolved/does not exist it throws FileSystemException('vfs.provider/copy-missing-file.error', sourceFile) — a guard copied from the VFS AbstractFileObject behavior.

Solutions

1. Verify the source S3 path/key spelling and case exactly matches the object.
2. Check exists() on the source before copyFrom() and fail gracefully if absent.
3. Confirm the source bucket and key are readable with the configured credentials.
4. Ensure the upstream step actually produced the source object before copying.

Example fix

// before
src.copyFrom( source, Selectors.SELECT_ALL );
// after
if ( !source.exists() ) {
  logger.warn( "Skipping copy; source missing: {}", source.getName() );
  return;
}
dst.copyFrom( source, Selectors.SELECT_ALL );

FileObject src = fs.resolveFile( srcUri );
if ( !src.exists() ) {
  throw new FileNotFoundException( "S3 source missing before copy: " + srcUri );
}
dst.copyFrom( src, Selectors.SELECT_ALL );

try {
  dst.copyFrom( src, Selectors.SELECT_ALL );
} catch ( FileSystemException e ) {
  if ( e.getMessage().contains( "copy-missing-file" ) ) {
    logger.warn( "Source vanished: {}", src.getName() );
    return; // treat as no-op instead of failing the job
  }
  throw e;
}

When it happens

Trigger: Calling copyFrom(file, selector) with a source FileObject that was deleted, never existed, has a typo'd S3 key, or whose attach failed so existence could not be established.

Common situations: Pipelines referencing an S3 key generated by a previous step that produced nothing, path case-sensitivity mismatches (S3 keys are case-sensitive), source bucket not accessible to the credentials.

Understand the failure class

Background: "File not found" and ENOENT errors: why libraries can't find a file that should exist — this error's family across 50 libraries.

Related errors

• vfs.provider/copy-file.error
• vfs.provider.s3/transfer.error
• vfs.provider.s3/transfer.interrupted
• vfs.provider.s3/transfer.null-argument
• Exception getting the list of buckets

AI-assisted analysis of pentaho/pentaho-kettle@f3058517a1 (2026-09-13). Data as JSON: /api/errors/66433f7e2a74f650. Report an issue: GitHub.

Appendix: source

Thrown at plugins/s3-vfs/core/src/main/java/org/pentaho/s3common/S3CommonFileObject.java:383

      }
    }
    return null;
  }

  /**
   * Copies the content of the specified file to this file
   * Uses server-side multipart copy on large files, if both files are S3FileObjects.
   * If the source is not an S3FileObject or is in a different region, it uses TransferManager to upload the content.
   * Supports both files and folders as source and destination. Empty source folders are not valid for S3 destination and will be ignored.
   *
   * @param file     The FileObject to copy.
   * @param selector The FileSelector.
   * @throws FileSystemException If an error occurs during the copy operation.
   */
  @Override
  public void copyFrom( final FileObject file, final FileSelector selector ) throws FileSystemException {
    if ( !FileObjectUtils.exists( file ) ) {
      throw new FileSystemException( "vfs.provider/copy-missing-file.error", file );
    }

    // Locate the files to copy across
    final ArrayList<FileObject> files = new ArrayList<>();
    file.findFiles( selector, false, files );

    // Copy everything across
    for ( final FileObject srcFile : files ) {
      // Skip folders - they will be created automatically when their files are copied, and empty folders do not exist in S3
      if ( srcFile.getType() == FileType.FOLDER ) {
        continue;
      }

      // Calculate the destination key preserving folder structure
      final S3CommonFileObject dstFile = calculateDestination( file, srcFile );

      // Copy the individual file
      copySingleFileFrom( srcFile, dstFile );

View on GitHub (pinned to f3058517a1)