{"record":{"id":"861c50073f014320","repo":"microsoft/markitdown","slug":"operation-failed-after-retries-attempts","errorCode":null,"errorMessage":"Operation failed after {retries} attempts.","messagePattern":"Operation failed after (.+?) attempts\\.","errorType":"exception","errorClass":"Exception","httpStatus":null,"severity":"error","filePath":"packages/markitdown/src/markitdown/converters/_youtube_converter.py","lineNumber":238,"sourceCode":"                if k == key:\n                    return json[k]\n                if result := self._findKey(v, key):\n                    return result\n        return None\n\n    def _retry_operation(self, operation, retries=3, delay=2):\n        \"\"\"Retries the operation if it fails.\"\"\"\n        attempt = 0\n        while attempt < retries:\n            try:\n                return operation()  # Attempt the operation\n            except Exception as e:\n                print(f\"Attempt {attempt + 1} failed: {e}\")\n                if attempt < retries - 1:\n                    time.sleep(delay)  # Wait before retrying\n                attempt += 1\n        # If all attempts fail, raise the last exception\n        raise Exception(f\"Operation failed after {retries} attempts.\")\n","sourceCodeStart":220,"sourceCodeEnd":239,"githubUrl":"https://github.com/microsoft/markitdown/blob/fd239d5d2be43d9b68329730206b9312c7d5a388/packages/markitdown/src/markitdown/converters/_youtube_converter.py#L220-L239","documentation":"YoutubeConverter._retry_operation() retries a callable up to `retries` times with `delay` seconds between attempts (printing each failure to stdout). When all attempts exhaust, it raises a bare Exception whose message only states the count — the original exception is neither chained (`from e`) nor included, so the root cause (network error, HTTP 470, extraction failure) is lost. Code that calls it (e.g. transcript or format fetching) surfaces only 'Operation failed after 3 attempts.'","triggerScenarios":"Converting a YouTube URL while youtube-transcript-api or yt-dlp operations fail every attempt: network blocks, YouTube rate-limiting/bot detection, age- or region-restricted videos, or invalid video IDs. Any transient-or-permanent failure repeated `retries` times (default 3) triggers this final generic Exception.","commonSituations":"Server IPs (cloud/CI) blocked by YouTube causing persistent failures; rate limits when converting many URLs in a loop; deleted/private videos; corporate proxies intercepting requests; the generic message making debugging hard because the printed stdout line is the only clue to the real error.","solutions":["Check the printed stdout lines ('Attempt N failed: ...') above the traceback — they contain the actual underlying error message","For YouTube blocking server IPs, route requests through a residential proxy or run from a network YouTube does not flag","Slow down between conversions (sleep/backoff between URLs) to avoid rate limiting","Verify the video is public, not age-restricted, and available in your region by opening the URL directly","As a library fix, chain the cause: `raise Exception(...) from e` so the traceback shows the root error"],"exampleFix":"# before (in _retry_operation)\n    raise Exception(f\"Operation failed after {retries} attempts.\")\n\n# after: preserve and chain the last cause\n    raise Exception(f\"Operation failed after {retries} attempts.\") from last_exception","handlingStrategy":"retry","validationCode":"import urllib.request, json\n\ndef youtube_video_available(video_id: str) -> bool:\n    try:\n        with urllib.request.urlopen(f\"https://www.youtube.com/oembed?url=https://www.youtube.com/watch?v={video_id}&format=json\", timeout=10) as r:\n            return r.status == 200\n    except Exception:\n        return False","typeGuard":null,"tryCatchPattern":"import time\nfrom markitdown import MarkItDown\n\nfor attempt in range(3):\n    try:\n        result = MarkItDown().convert(\"https://www.youtube.com/watch?v=VIDEO_ID\")\n        break\n    except Exception as e:\n        # message is generic ('Operation failed after N attempts.'); real cause is on stdout\n        if attempt == 2:\n            logger.error(\"youtube conversion failed; check 'Attempt N failed' stdout lines\")\n            raise\n        time.sleep(2 ** attempt)","preventionTips":["Capture stdout/stderr of the process: _retry_operation prints the real per-attempt errors there","Rate-limit and add jittered backoff between YouTube conversions to avoid bot detection","Pre-validate URLs with the oembed endpoint to skip deleted/private/region-locked videos before conversion","Run YouTube conversions from residential IPs or a rotating proxy when server IPs get blocked"],"tags":["youtube","retry","network","rate-limiting","lost-cause"],"backgroundTag":null,"analyzedSha":"fd239d5d2be43d9b68329730206b9312c7d5a388","analyzedAt":"2026-08-14T15:47:51.745Z","schemaVersion":2},"datasetVersion":"2026-08-16T03:17:38.424Z"}