{"record":{"id":"4e86fdb833cc5adb","repo":"calesthio/OpenMontage","slug":"could-not-find-download-url-on-dareful-page-deta","errorCode":null,"errorMessage":"Could not find download URL on Dareful page: {detail_url}","messagePattern":"Could not find download URL on Dareful page: (.+?)","errorType":"exception","errorClass":"ValueError","httpStatus":null,"severity":"warning","filePath":"tools/video/stock_sources/dareful.py","lineNumber":165,"sourceCode":"                href = a.get(\"href\", \"\")\n                text = (a.get_text(strip=True) or \"\").lower()\n                if any(ext in href.lower() for ext in [\".mp4\", \".mov\", \".webm\"]):\n                    download_url = href\n                    break\n                if \"download\" in text and href:\n                    download_url = href\n                    break\n\n            # Check video elements\n            if not download_url:\n                for source in soup.select(\"video source[src], video[src]\"):\n                    src = source.get(\"src\", \"\")\n                    if src:\n                        download_url = src\n                        break\n\n            if not download_url:\n                raise ValueError(f\"Could not find download URL on Dareful page: {detail_url}\")\n\n            if not download_url.startswith(\"http\"):\n                download_url = f\"{_BASE_URL}{download_url}\"\n\n            return self._stream_download(download_url, out_path)\n\n        except Exception as e:\n            raise RuntimeError(f\"Dareful download failed for {detail_url}: {e}\") from e\n\n    def _stream_download(self, url: str, out_path: Path) -> Path:\n        import requests\n\n        with requests.get(\n            url, stream=True, timeout=180,\n            headers={\"User-Agent\": \"OpenMontage/1.0\"},\n        ) as r:\n            r.raise_for_status()\n            with open(out_path, \"wb\") as f:","sourceCodeStart":147,"sourceCodeEnd":183,"githubUrl":"https://github.com/calesthio/OpenMontage/blob/95e1c3d0ab93482159818560f6a8c8e866b9139f/tools/video/stock_sources/dareful.py#L147-L183","documentation":"ValueError raised by the Dareful adapter's download() when scraping the clip's detail page finds neither an anchor href matching its download-link selectors nor a <video><source src> element. Dareful has no public JSON API, so the adapter parses HTML; failure means the page layout changed, the clip is member-gated, or the page failed to render the expected elements.","triggerScenarios":"Dareful redesigns their detail-page template; the clip requires login (download button only present for members); a Cloudflare/interstitial page is served instead of the real detail page; the detail_url itself 404s or redirects.","commonSituations":"Sites like Dareful changing markup without notice (classic scraper fragility); geo-blocks or bot detection returning alternate HTML; stale detail_url captured in an earlier search and expired since.","solutions":["Open the detail_url in a browser and confirm a direct video link still exists on the page.","If gated, fetch the file URL yourself while authenticated and stream it directly, bypassing the adapter's scrape.","Fall back to another stock source (Pexels, Archive.org) for that clip — use get_source() on an alternate adapter.","If Dareful's markup changed, update the CSS selectors in dareful.py to match the new page structure."],"exampleFix":"# before\npath = dareful.download(candidate, out)\n\n# after\ntry:\n    path = dareful.download(candidate, out)\nexcept (ValueError, RuntimeError):\n    path = get_source(\"pexels\").download(alternate_candidate, out)","handlingStrategy":"fallback","validationCode":null,"typeGuard":null,"tryCatchPattern":"try:\n    path = dareful.download(candidate, out_path)\nexcept (ValueError, RuntimeError) as e:\n    logger.warning(\"dareful scrape failed: %s\", e)\n    path = alternate_source.download(alternate_candidate, out_path)","preventionTips":["Prefer API-based stock sources (Pexels) for reliability; treat scraped sources as best-effort.","Refresh search results close to download time so detail pages are current."],"tags":["stock-footage","dareful","scraping","fragile","download"],"backgroundTag":null,"analyzedSha":"95e1c3d0ab93482159818560f6a8c8e866b9139f","analyzedAt":"2026-08-15T06:31:20.014Z","schemaVersion":2},"datasetVersion":"2026-08-15T17:31:12.345Z"}