windmill-labs/windmill · error · Exception

Could not delete file from S3

Error message

Could not delete file from S3

What it means

Generic fallback in delete_s3_object: the deletion failed with an exception that is not an httpx.HTTPStatusError — typically a connection error, timeout, or other client-side/transport failure. Unlike the status-error path, no URL or status detail is included.

Source

Thrown at python-client/wmill/wmill/client.py:1054

        """
        s3object = parse_s3_object(s3object)
        query_params: Dict[str, Any] = {"file_key": s3object["s3"]}
        if s3_resource_path is not None and s3_resource_path != "":
            query_params["s3_resource_path"] = s3_resource_path
        if "storage" in s3object and s3object["storage"] is not None:
            query_params["storage"] = s3object["storage"]
        try:
            resp = self.client.delete(
                f"/w/{self.workspace}/job_helpers/delete_s3_file",
                params=query_params,
            )
            resp.raise_for_status()
        except httpx.HTTPStatusError as err:
            error = f"{err.request.url}: {err.response.status_code}, {err.response.text}"
            logger.error(error)
            raise Exception(error)
        except Exception as e:
            raise Exception("Could not delete file from S3") from e

    def sign_s3_objects(
        self, s3_objects: list[S3Object | str], expiry_secs: int | None = None
    ) -> list[S3Object]:
        """Sign S3 objects for use by anonymous users in public apps.

        Args:
            s3_objects: List of S3 objects to sign
            expiry_secs: How long the signature stays valid, in seconds
                (defaults to 43200 = 12h, clamped to [60, 604800])

        Returns:
            List of signed S3 objects
        """
        return self.post(
            f"/w/{self.workspace}/apps/sign_s3_objects",
            json=_sign_s3_objects_body(list(map(parse_s3_object, s3_objects)), expiry_secs),
        ).json()

View on GitHub (pinned to e474e8803c)

Solutions

1. Log and inspect the chained exception (e.__cause__) for the transport-level cause
2. Add retry with exponential backoff for transient errors
3. Verify backend URL and network reachability
4. Check TLS/proxy configuration on self-hosted instances

Example fix

// before
wm.delete_s3_object(s3obj)
// after
import time
for attempt in range(3):
    try:
        wm.delete_s3_object(s3obj)
        break
    except Exception as e:
        if "Could not delete" in str(e) and attempt < 2:
            time.sleep(2 ** attempt)
        else:
            raise

import httpx
try:
    httpx.get(f"{wm.get_client().base_url}/api/version", timeout=5)
except httpx.HTTPError:
    raise RuntimeError("Windmill backend unreachable")

import time
for attempt in range(3):
    try:
        wm.delete_s3_object(s3obj)
        break
    except Exception as e:
        if attempt == 2 or "Could not delete" not in str(e):
            raise
        time.sleep(2 ** attempt)

When it happens

Trigger: Network outage or DNS failure reaching the backend, request timeout, SSL/TLS errors, or httpx client construction failures during delete_s3_object.

Common situations: Self-hosted backend temporarily down, VPN/proxy blocking the connection, transient network blips in scheduled jobs.

Related errors

• Giving up polling job ${jobId} after ${MAX_CONSECUTIVE_POLL_
• Failed to poll dependencies job ${jobId}: ${e?.message ?? e}
• Failed to load S3 file: ${response.status} ${response.status
• Could not write file to S3
• error writing file to {path}: {e:#}

AI-assisted analysis of windmill-labs/windmill@e474e8803c (2026-09-03). Data as JSON: /api/errors/74b5cac9fa4095cc. Report an issue: GitHub.