NanmiCoder/MediaCrawler · error · ValueError

Unable to parse video ID from URL

Error message

Unable to parse video ID from URL: {url}

What it means

ValueError from parse_video_info_from_url (media_platform/douyin/help.py) when the URL yields no extractable aweme id: the modal_id query param is absent AND the /video/(\d+) regex does not match. The parser accepts modal_id links (note/search pages with modal_id) and standard /video/<digits> URLs; everything else — short links (v.douyin.com), live or note image URLs without modal_id, non-numeric ids — fails here.

Solutions

  1. Expand v.douyin.com short links (HTTP redirect) to the final www.douyin.com URL before parsing.
  2. Pass the bare numeric aweme id or the canonical https://www.douyin.com/video/<id> URL.
  3. Pre-filter URLs through a regex (/video/\d+ or modal_id=) and skip/log non-matching ones in batch jobs.

Example fix

# before
parse_video_info_from_url('https://v.douyin.com/iAbCdEf/')

# after
resolved = await expand_short_link('https://v.douyin.com/iAbCdEf/')  # -> https://www.douyin.com/video/7525082444551310602
parse_video_info_from_url(resolved)
Defensive patterns

Strategy: validation

Validate before calling

import re
DY_RE = re.compile(r'/video/\d+')
def has_douyin_video_id(u: str) -> bool:
    return bool(DY_RE.search(u)) or 'modal_id=' in u

Type guard

def is_parseable_dy_video(url: str) -> bool:
    return bool(re.search(r'/video/\d+', url)) or 'modal_id=' in url

Try / catch

try:
    info = parse_video_info_from_url(url)
except ValueError:
    logger.warning(f'skipping unparseable douyin url: {url}')
    continue

Prevention

When it happens

Trigger: Passing a raw 'https://v.douyin.com/xxxx/' share short link that 302s to the real URL; a note page URL without a modal_id param; an empty string or a bare non-numeric id; URLs where the digits sit under a different path like /slide/.

Common situations: Users pasting the share-text link from the Douyin mobile app; crawling mixed feeds that include image notes and live links; upstream URL format changes adding new path shapes.

Understand the failure class

Related errors

AI-assisted analysis of NanmiCoder/MediaCrawler@d6f7c5bb90 (2026-08-15). Data as JSON: /api/errors/92dd19bc4850f32d. Report an issue: GitHub.

Appendix: source

Thrown at media_platform/douyin/help.py:138

    # Check if it's a short link (v.douyin.com)
    if "v.douyin.com" in url or url.startswith("http") and len(url) < 50 and "video" not in url:
        return VideoUrlInfo(aweme_id="", url_type="short")  # Requires client parsing

    # Try to extract modal_id from URL parameters
    params = extract_url_params_to_dict(url)
    modal_id = params.get("modal_id")
    if modal_id:
        return VideoUrlInfo(aweme_id=modal_id, url_type="modal")

    # Extract ID from standard video URL: /video/number
    video_pattern = r'/video/(\d+)'
    match = re.search(video_pattern, url)
    if match:
        aweme_id = match.group(1)
        return VideoUrlInfo(aweme_id=aweme_id, url_type="normal")

    raise ValueError(f"Unable to parse video ID from URL: {url}")


def parse_creator_info_from_url(url: str) -> CreatorUrlInfo:
    """
    Parse creator ID (sec_user_id) from Douyin creator homepage URL
    Supports the following formats:
    1. Creator homepage: https://www.douyin.com/user/MS4wLjABAAAATJPY7LAlaa5X-c8uNdWkvz0jUGgpw4eeXIwu_8BhvqE?from_tab_name=main
    2. Pure ID: MS4wLjABAAAATJPY7LAlaa5X-c8uNdWkvz0jUGgpw4eeXIwu_8BhvqE

    Args:
        url: Douyin creator homepage link or sec_user_id
    Returns:
        CreatorUrlInfo: Object containing creator ID
    """
    # If it's a pure ID format (usually starts with MS4wLjABAAAA), return directly
    if url.startswith("MS4wLjABAAAA") or (not url.startswith("http") and "douyin.com" not in url):
        return CreatorUrlInfo(sec_user_id=url)

View on GitHub (pinned to d6f7c5bb90)