apache/superset · warning · AsyncQueryJobException

Job not found or already completed

Error message

Job not found or already completed

What it means

AsyncQueryJobException raised in the job-cancellation path of the async query manager: when the per-job registry record (Redis key '<stream-prefix>job-cancel:<job_id>') is missing (cache.get returns None), the job is unknown — either it never existed, or it already reached a terminal state and cleaned up its record. Cancellation requires a live registry entry so it can be flagged and the Celery task revoked.

Source

Thrown at superset/async_events/async_query_manager.py:437

        Authorize and cancel a running async job.

        The caller's ``channel_id`` and ``user_id`` (resolved server-side from
        the request, never taken from the client) must match the job's original
        owner. The terminal ``STATUS_CANCELLED`` event is emitted here rather
        than by the worker, which never runs for a task revoked while it was
        still queued; the worker only logs the cancellation it is told about
        through the flag, so a job still gets exactly one terminal event.

        :raises AsyncQueryJobException: the job is unknown or already terminal
        :raises AsyncQueryTokenException: the caller does not own the job
        """
        if not self._cache:
            raise CacheBackendNotInitialized("Cache backend not initialized")

        key = self._job_registry_key(job_id)
        raw = self._cache.get(key)
        if raw is None:
            raise AsyncQueryJobException("Job not found or already completed")

        record = json.loads(raw)
        if record.get("channel_id") != channel_id or record.get("user_id") != user_id:
            raise AsyncQueryTokenException("Not authorized to cancel this job")

        # Flag before revoking so the worker's timeout handler, which may fire
        # almost immediately, reliably sees the cancellation. Write only if the
        # key still exists (``xx``): if the job finished and cleared its record
        # between the read above and here, don't recreate a stale record or
        # revoke a task that is already gone — report it as not found instead.
        flagged = self._cache.set(
            key,
            json.dumps({**record, "cancelled": True}),
            ex=self._jwt_expiration_seconds or None,
            xx=True,
        )
        if not flagged:
            raise AsyncQueryJobException("Job not found or already completed")

View on GitHub (pinned to f4587218dd)

Solutions

1. Treat this as benign in races: catch AsyncQueryJobException for 'already completed' and surface 'job already finished' to the user instead of an error.
2. Verify the job_id matches the one returned in the job metadata from the original request (check job_id/user_id/channel_id).
3. If jobs legitimately outlive the registry TTL, raise GLOBAL_ASYNC_QUERIES_JWT_EXPIRATION so registry entries persist for the full expected job duration.

Example fix

# before
manager.cancel_job(job_id, channel_id, user_id)  # may raise mid-request

# after
from superset.async_events.async_query_manager import AsyncQueryJobException
try:
    manager.cancel_job(job_id, channel_id, user_id)
except AsyncQueryJobException:
    logger.info("Job %s already terminal; ignoring cancel", job_id)

raw = cache.get(f"{stream_prefix}job-cancel:{job_id}")
if raw is None:
    skip_cancel("job unknown or already terminal")

try:
    manager.cancel_job(job_id, channel_id, user_id)
except AsyncQueryJobException:
    # unknown or terminal: not an error for the user
    log.info("cancel skipped: job %s already completed", job_id)

When it happens

Trigger: Calling the cancel endpoint (DELETE on the async job resource) with a job_id that was mistyped, already finished successfully, already errored, was already cancelled, or whose registry entry expired (TTL is jwt_expiration_seconds). Also double-clicking a cancel button — the second request finds the record gone.

Common situations: Frontends racing: the job completes normally right as the user hits cancel; retrying a cancel request that already succeeded; long-running jobs whose registry entry TTL (tied to JWT expiration) is shorter than the job duration; stale UI showing a job that the worker already finalized.

Related errors

• Unsupported cache backend configuration
• Cache backends (CACHE_CONFIG, DATA_CACHE_CONFIG) must be con
• Not authorized to cancel this job
• Please provide a JWT secret at least 32 bytes long
• Token not preset

AI-assisted analysis of apache/superset@f4587218dd (2026-08-14). Data as JSON: /api/errors/cd18f9c9ce5fe116. Report an issue: GitHub.