mlflow/mlflow · error · MlflowException

INVALID_PARAMETER_VALUE

INVALID_PARAMETER_VALUE

Error message

The installed databricks-sdk version does not support uploading files larger than 5GB. Please upgrade the databricks-sdk package to version >= 0.45.0.

What it means

DatabricksSdkArtifactRepo.log_artifact rejects files larger than 5GB when the installed databricks-sdk lacks large-file upload support (Workspace file upload APIs for >5GB arrived in databricks-sdk 0.45.0). The repository checks file size up front and raises INVALID_PARAMETER_VALUE to avoid a doomed upload.

Source

Thrown at mlflow/store/artifact/databricks_sdk_artifact_repo.py:82

    def files_api(self) -> "FilesAPI":
        return self.wc.files

    def _is_dir(self, path: str) -> bool:
        from databricks.sdk.errors.platform import NotFound

        try:
            self.files_api.get_directory_metadata(path)
        except NotFound:
            return False
        return True

    def full_path(self, artifact_path: str | None) -> str:
        return f"{self.artifact_uri}/{artifact_path}" if artifact_path else self.artifact_uri

    def log_artifact(self, local_file: str, artifact_path: str | None = None) -> None:
        is_large_file = Path(local_file).stat().st_size > 5 * (1024**3)
        if is_large_file and not self._supports_large_file_uploads:
            raise MlflowException.invalid_parameter_value(
                "The installed databricks-sdk version does not support uploading files larger "
                "than 5GB. Please upgrade the databricks-sdk package to version >= 0.45.0."
            )

        with open(local_file, "rb") as f:
            name = Path(local_file).name
            self.files_api.upload(
                self.full_path(posixpath.join(artifact_path, name) if artifact_path else name),
                f,
                overwrite=True,
            )

    def log_artifacts(self, local_dir: str, artifact_path: str | None = None) -> None:
        local_dir = Path(local_dir).resolve()
        futures: list[Future[None]] = []
        with self._create_thread_pool() as executor:
            for f in local_dir.rglob("*"):
                if not f.is_file():

View on GitHub (pinned to 6a27f2decc)

Solutions

  1. Upgrade databricks-sdk to >= 0.45.0 (pip install -U 'databricks-sdk>=0.45.0').
  2. If the SDK cannot be upgraded, split or compress the file so it is under 5GB, or store large files in DBFS/Volumes or object storage (S3/ADLS) and log a reference.
  3. Check installed version with databricks.sdk version metadata before large uploads and fail fast with a clear message.

Example fix

// before (requirements.txt)
databricks-sdk==0.30.0

// after
databricks-sdk>=0.45.0
Defensive patterns

Strategy: validation

Validate before calling

import importlib.metadata
from pathlib import Path
sdk_version = tuple(int(x) for x in importlib.metadata.version("databricks-sdk").split(".")[:2])
assert Path(local_file).stat().st_size <= 5 * 1024**3 or sdk_version >= (0, 45), "upgrade databricks-sdk>=0.45.0"

Type guard

def supports_large_uploads(size_bytes: int, sdk_version: tuple[int, int]) -> bool:
    return size_bytes <= 5 * 1024**3 or sdk_version >= (0, 45)

Try / catch

try:
    repo.log_artifact(large_file)
except MlflowException as e:
    if e.error_code == "INVALID_PARAMETER_VALUE" and "5GB" in str(e):
        log_to_object_storage_and_reference(large_file)  # e.g. DBFS/Volumes or S3 + log URI

Prevention

When it happens

Trigger: log_artifact(local_file) where Path(local_file).stat().st_size > 5*1024**3 and self._supports_large_file_uploads is False (databricks-sdk < 0.45.0 installed).

Common situations: Old databricks-sdk pinned in requirements while logging large model checkpoints (multi-GB weights, datasets); environments where uv/pip resolved an older SDK due to constraints; Docker images with stale SDK versions.

Related errors

AI-assisted analysis of mlflow/mlflow@6a27f2decc (2026-08-29). Data as JSON: /api/errors/393bea98191dbcb4. Report an issue: GitHub.