unclecode/crawl4ai · warning · HTTPException
Task not found
Error message
Task not found
What it means
HTTP 404 'Task not found' from the task-status handler (deploy/docker/api.py:497) when Redis hgetall(f"task:{task_id}") returns an empty hash — the task key does not exist. Common causes: task already completed and auto-cleaned, TTL expired, wrong/typo'd task_id, or Redis flushed/restarted without persistence. It is deliberately indistinguishable from the ownership 404 at 507.
Source
Thrown at deploy/docker/api.py:497
async def handle_task_status(
redis: aioredis.Redis,
task_id: str,
base_url: str,
*,
keep: bool = False,
requester: Optional[str] = None,
is_admin: bool = False,
) -> JSONResponse:
"""Handle task status check requests.
Enforces ownership: a task records the `owner` (principal sub) that created
it; a different requester gets 404 (not 403, so task existence is not
revealed). Admin-scope principals may read any task.
"""
task = await redis.hgetall(f"task:{task_id}")
if not task:
raise HTTPException(
status_code=status.HTTP_404_NOT_FOUND,
detail="Task not found"
)
task = decode_redis_hash(task)
owner = task.get("owner")
if owner and not is_admin and owner != requester:
# Do not leak existence of someone else's task.
raise HTTPException(
status_code=status.HTTP_404_NOT_FOUND,
detail="Task not found"
)
response = create_task_response(task, task_id, base_url)
if task["status"] in [TaskStatus.COMPLETED, TaskStatus.FAILED]:
if not keep and should_cleanup_task(task["created_at"]):View on GitHub (pinned to 7e80152142)
Solutions
- Poll frequently enough to observe the terminal status before auto-cleanup, or request keep=True so the record persists.
- Treat 404 on a previously-seen task id as 'finished & reaped' — record the last observed status client-side.
- Persist Redis (appendonly/volume) so task keys survive restarts.
- Double-check the task_id is exactly the one returned at submission.
Example fix
# before
status = client.get(f"{base}/task/{task_id}").json()
# after
last = {}
while True:
r = client.get(f"{base}/task/{task_id}")
if r.status_code == 404:
print("final:", last or "unknown (reaped before first poll)")
break
last = r.json()
if last["status"] in ("completed", "failed"):
break
time.sleep(2) Defensive patterns
Strategy: retry
Validate before calling
def task_id_plausible(task_id: str) -> bool:
return isinstance(task_id, str) and 8 <= len(task_id) <= 64 and task_id.isalnum() Try / catch
r = await client.get(f"/task/{task_id}")
if r.status_code == 404:
if task_id in seen_task_ids:
return last_status[task_id] # finished & reaped earlier
raise TaskLostError(task_id) # never existed / wrong id Prevention
- Poll at short intervals until terminal status so cleanup never outruns you
- Pass keep=true for tasks you must audit later
- Persist the last observed status client-side for every task_id
- Run Redis with persistence enabled so task records survive restarts
When it happens
Trigger: Polling GET /task/{id} after the task finished and should_cleanup_task deleted it (server deletes completed/failed tasks on read when keep=False and age exceeds threshold); polling long after submission so TTL expired; using a task_id from a previous server/Redis instance.
Common situations: Clients that poll infrequently and miss the terminal state before cleanup; Redis restart without volume; race between two pollers where one read triggers deletion and the other then 404s.
Related errors
AI-assisted analysis of unclecode/crawl4ai@7e80152142 (2026-08-14).
Data as JSON: /api/errors/4d5625349499d33e.
Report an issue: GitHub.