{"record":{"id":"12246e5647f48ae3","repo":"soimort/you-get","slug":"not-found-total-count-in-html","errorCode":null,"errorMessage":"not found total count in html","messagePattern":"not found total count in html","errorType":"exception","errorClass":"AssertionError","httpStatus":null,"severity":"error","filePath":"src/you_get/extractors/lrts.py","lineNumber":24,"sourceCode":"from ..common import *\nfrom ..util import log, term\n\ndef lrts_download(url, output_dir='.', merge=True, info_only=False, **kwargs):\n    html = get_html(url)\n    args = kwargs.get('args')\n    if not args: args = {}\n    matched = re.search(r\"/book/(\\d+)\", url)\n    if not matched:\n        raise AssertionError(\"not found book number: %s\" % url)\n    book_no = matched.group(1)\n    book_title = book_no\n    matched = re.search(r\"<title>([^-]*)[-](.*)[,](.*)</title>\", html)\n    if matched:\n        book_title = matched.group(1)\n\n    matched = re.search(r\"var totalCount='(\\d+)'\", html)\n    if not matched:\n        raise AssertionError(\"not found total count in html\")\n    total_count = int(matched.group(1))\n    log.i('%s total: %s' % (book_title, total_count))\n    first_page = 0\n    if ('first' in args and args.first!= None):\n        first_page = int(args.first)\n\n    page_size = 10\n    if ('page_size' in args and args.page_size != None):\n        page_size = int(args.page_size)\n    last_page = (total_count // page_size) + 1\n    if ('last' in args and args.last != None):\n        last_page = int(args.last)\n\n    log.i('page size is %s, page from %s to %s' % (page_size, first_page, last_page))\n    headers = {\n      'Referer': url\n    }\n    items = []","sourceCodeStart":6,"sourceCodeEnd":42,"githubUrl":"https://github.com/soimort/you-get/blob/049548f3f3f35e67ba8d3181c71fdc71d11cf260/src/you_get/extractors/lrts.py#L6-L42","documentation":"After fetching the lrts.me book page, the extractor scrapes the total chapter count via the literal pattern r\"var totalCount='(\\d+)'\" from the raw HTML. If the markup no longer embeds that JavaScript variable, it raises AssertionError('not found total count in html') — the extractor cannot compute pagination (last_page = total_count // page_size + 1) without it.","triggerScenarios":"The fetched HTML is not the expected book page: a login/region/captcha interstitial, a 200-status error page, or a site redesign that renamed or reformatted the totalCount variable (quotes, spacing, JSON-ification).","commonSituations":"Site frontend rewrite moving the value into a JSON blob or script bundle; being geo-blocked or rate-limited so get_html returns an error shell; the book being removed so the page renders without chapter data.","solutions":["Print/inspect the HTML returned by get_html(url) — confirm you got the real book page and not a block/login page","Locate the new home of the total count in the page source (search for 'totalCount' or the chapter count) and update the regex at src/you_get/extractors/lrts.py:24","If the value moved to an AJAX endpoint, fetch that endpoint instead of scraping HTML","Pass explicit --first/--last page args once pagination is understood, so the count scrape can be bypassed"],"exampleFix":"# before\nmatched = re.search(r\"var totalCount='(\\d+)'\", html)\nif not matched:\n    raise AssertionError(\"not found total count in html\")\n\n# after (tolerate double quotes and JSON style)\nmatched = re.search(r\"totalCount['\\\"]?\\s*[:=]\\s*['\\\"]?(\\d+)\", html)\nif not matched:\n    raise AssertionError(\"not found total count in html\")","handlingStrategy":"validation","validationCode":"import re\n\ndef lrts_page_has_count(html):\n    return re.search(r\"var totalCount='(\\d+)'\", html) is not None","typeGuard":null,"tryCatchPattern":"try:\n    lrts_download(url, ...)\nexcept AssertionError as e:\n    if 'total count' in str(e):\n        print('page layout changed or a block page was served; inspect HTML')\n    else:\n        raise","preventionTips":["Spot-check that get_html returns the real book page (title tag, chapter markup) before scraping","Pin automated scrapers to a known site version or add a fallback pattern for totalCount","Watch for login/geo interstitials that return HTTP 200 with an error shell"],"tags":["html-scraping","lrts","site-redesign","regex"],"backgroundTag":null,"analyzedSha":"049548f3f3f35e67ba8d3181c71fdc71d11cf260","analyzedAt":"2026-08-15T03:58:15.069Z","schemaVersion":2},"datasetVersion":"2026-08-15T17:31:12.345Z"}