What causes "Unable to parse S3 URL: {https_url}" in BerriAI/litellm?

Verify s3_bucket_name / object naming so the final URL is endpoint/bucket/key with a non-empty key

BerriAI/litellm · error · ValueError

Unable to parse S3 URL

Error message

Unable to parse S3 URL: {https_url}

What it means

After the S3 PUT, transform_create_file_response converts the upload URL back into the s3:// URI that becomes the file id. Two regexes handle path-style (https://s3.region.amazonaws.com/bucket/key) and virtual-hosted (https://bucket.s3.region.amazonaws.com/key) amazonaws URLs; a fallback parses bucket/key from the URL path. If the path has fewer than two segments (no /bucket/key), parsing fails.

Solutions

Verify s3_bucket_name / object naming so the final URL is endpoint/bucket/key with a non-empty key
For custom endpoints, use path-style addressing so bucket and key both appear in the URL path
Log litellm_params['upload_url'] after create to see the exact URL the parser rejected
If the URL looks valid but non-amazonaws, report it as a parser gap at github.com/BerriAI/litellm/issues

Example fix

# before: proxy config
general_settings:
  s3_bucket_name: "my-bucket"   # object key generated empty due to misconfigured upload_url

# after: ensure upload URL is path-style with bucket+key
# e.g. https://s3.us-west-2.amazonaws.com/my-bucket/path/to/file.jsonl

Defensive patterns

Strategy: try-catch

Validate before calling

from urllib.parse import urlparse

def upload_url_is_parseable(url: str) -> bool:
    p = urlparse(url)
    return p.hostname and len(p.path.lstrip("/").split("/", 1)) >= 2 and all(p.path.lstrip("/").split("/", 1))

Try / catch

try:
    file_obj = await client.files.create(file=f, purpose="batch")
except ValueError as e:
    if "Unable to parse S3 URL" in str(e):
        # inspect litellm_params['upload_url']: custom endpoint or empty object key
        raise RuntimeError(f"upload URL not in endpoint/bucket/key form: {e}") from e
    raise

Prevention

Configure custom S3 endpoints to use path-style URLs so bucket and key are both in the path
Ensure bucket and object-key configuration always yields a non-empty key
Smoke-test one create_file per environment to catch endpoint-shape issues at deploy time

When it happens

Trigger: An upload URL whose path lacks bucket+key: an empty object key (bucket name configured but object name empty), a custom s3_endpoint_url that returns a host-only or bucket-only URL, or a non-amazonaws virtual-hosted endpoint where the bucket lives in the hostname and the path holds only the key.

Common situations: Custom S3-compatible endpoints (MinIO, LocalStack, corporate S3 gateways) that do not match the amazonaws regexes; trailing configuration mistakes where s3_bucket_name swallows the key; proxy configs rewriting the upload URL.

Understand the failure class

Parsing and encoding errors: unexpected token, malformed input — why parsers reject input and how to find the real culprit.

Related errors

AI-assisted analysis of BerriAI/litellm@77b7c6c40c (2026-08-18). Data as JSON: /api/errors/fae6818f7632da4c. Report an issue: GitHub.

Appendix: source

Thrown at litellm/llms/bedrock/files/transformation.py:1129

            key = unquote(key)
            s3_uri = f"s3://{bucket}/{key}"
        elif match2:
            # Pattern: https://bucket.s3.region.amazonaws.com/key
            bucket, region, key = match2.groups()
            key = unquote(key)
            s3_uri = f"s3://{bucket}/{key}"
        else:
            # Fallback: try to extract bucket and key from URL path
            from urllib.parse import urlparse

            parsed: Final = urlparse(https_url)
            path_parts: Final = parsed.path.lstrip("/").split("/", 1)
            if len(path_parts) >= 2:
                bucket, key = path_parts[0], path_parts[1]
                key = unquote(key)
                s3_uri = f"s3://{bucket}/{key}"
            else:
                raise ValueError(f"Unable to parse S3 URL: {https_url}")

        # Extract filename from key
        filename: Final = key.split("/")[-1] if "/" in key else key

        return s3_uri, filename

    def transform_create_file_response(
        self,
        model: str | None,
        raw_response: Response,
        logging_obj: LiteLLMLoggingObj,
        litellm_params: dict,
    ) -> OpenAIFileObject:
        """
        Transform S3 File upload response into OpenAI-style FileObject
        """
        # Use the actual upload URL that was used for the S3 upload
        upload_url: Final = litellm_params.get("upload_url")

View on GitHub (pinned to 77b7c6c40c)