NanmiCoder/MediaCrawler · error · ValueError

Unable to parse video ID from URL: {url}

Error message

Unable to parse video ID from URL: {url}

What it means

ValueError from parse_video_info_from_url (media_platform/douyin/help.py) when the URL yields no extractable aweme id: the modal_id query param is absent AND the /video/(\d+) regex does not match. The parser accepts modal_id links (note/search pages with modal_id) and standard /video/<digits> URLs; everything else — short links (v.douyin.com), live or note image URLs without modal_id, non-numeric ids — fails here.

Source

Thrown at media_platform/douyin/help.py:138

    # Check if it's a short link (v.douyin.com)
    if "v.douyin.com" in url or url.startswith("http") and len(url) < 50 and "video" not in url:
        return VideoUrlInfo(aweme_id="", url_type="short")  # Requires client parsing

    # Try to extract modal_id from URL parameters
    params = extract_url_params_to_dict(url)
    modal_id = params.get("modal_id")
    if modal_id:
        return VideoUrlInfo(aweme_id=modal_id, url_type="modal")

    # Extract ID from standard video URL: /video/number
    video_pattern = r'/video/(\d+)'
    match = re.search(video_pattern, url)
    if match:
        aweme_id = match.group(1)
        return VideoUrlInfo(aweme_id=aweme_id, url_type="normal")

    raise ValueError(f"Unable to parse video ID from URL: {url}")


def parse_creator_info_from_url(url: str) -> CreatorUrlInfo:
    """
    Parse creator ID (sec_user_id) from Douyin creator homepage URL
    Supports the following formats:
    1. Creator homepage: https://www.douyin.com/user/MS4wLjABAAAATJPY7LAlaa5X-c8uNdWkvz0jUGgpw4eeXIwu_8BhvqE?from_tab_name=main
    2. Pure ID: MS4wLjABAAAATJPY7LAlaa5X-c8uNdWkvz0jUGgpw4eeXIwu_8BhvqE

    Args:
        url: Douyin creator homepage link or sec_user_id
    Returns:
        CreatorUrlInfo: Object containing creator ID
    """
    # If it's a pure ID format (usually starts with MS4wLjABAAAA), return directly
    if url.startswith("MS4wLjABAAAA") or (not url.startswith("http") and "douyin.com" not in url):
        return CreatorUrlInfo(sec_user_id=url)

View on GitHub (pinned to d6f7c5bb90)

Solutions

  1. Expand v.douyin.com short links (HTTP redirect) to the final www.douyin.com URL before parsing.
  2. Pass the bare numeric aweme id or the canonical https://www.douyin.com/video/<id> URL.
  3. Pre-filter URLs through a regex (/video/\d+ or modal_id=) and skip/log non-matching ones in batch jobs.

Example fix

# before
parse_video_info_from_url('https://v.douyin.com/iAbCdEf/')

# after
resolved = await expand_short_link('https://v.douyin.com/iAbCdEf/')  # -> https://www.douyin.com/video/7525082444551310602
parse_video_info_from_url(resolved)
Defensive patterns

Strategy: validation

Validate before calling

import re
DY_RE = re.compile(r'/video/\d+')
def has_douyin_video_id(u: str) -> bool:
    return bool(DY_RE.search(u)) or 'modal_id=' in u

Type guard

def is_parseable_dy_video(url: str) -> bool:
    return bool(re.search(r'/video/\d+', url)) or 'modal_id=' in url

Try / catch

try:
    info = parse_video_info_from_url(url)
except ValueError:
    logger.warning(f'skipping unparseable douyin url: {url}')
    continue

Prevention

When it happens

Trigger: Passing a raw 'https://v.douyin.com/xxxx/' share short link that 302s to the real URL; a note page URL without a modal_id param; an empty string or a bare non-numeric id; URLs where the digits sit under a different path like /slide/.

Common situations: Users pasting the share-text link from the Douyin mobile app; crawling mixed feeds that include image notes and live links; upstream URL format changes adding new path shapes.

Understand the failure class

Related errors

AI-assisted analysis of NanmiCoder/MediaCrawler@d6f7c5bb90 (2026-08-15). Data as JSON: /api/errors/92dd19bc4850f32d. Report an issue: GitHub.