NanmiCoder/MediaCrawler · error · ValueError

Unable to parse creator ID from URL: {url}

Error message

Unable to parse creator ID from URL: {url}

What it means

ValueError from parse_creator_info_from_url (media_platform/douyin/help.py) when the input is neither a bare sec_user_id (prefix 'MS4wLjABAAAA', or any non-http string not containing douyin.com) nor a URL containing /user/<segment>. Douyin creator ids are the long MS4w... sec_user_id strings embedded in /user/ paths, so usernames, homepages with other path shapes, or malformed links fall through.

Source

Thrown at media_platform/douyin/help.py:164

    2. Pure ID: MS4wLjABAAAATJPY7LAlaa5X-c8uNdWkvz0jUGgpw4eeXIwu_8BhvqE

    Args:
        url: Douyin creator homepage link or sec_user_id
    Returns:
        CreatorUrlInfo: Object containing creator ID
    """
    # If it's a pure ID format (usually starts with MS4wLjABAAAA), return directly
    if url.startswith("MS4wLjABAAAA") or (not url.startswith("http") and "douyin.com" not in url):
        return CreatorUrlInfo(sec_user_id=url)

    # Extract sec_user_id from creator homepage URL: /user/xxx
    user_pattern = r'/user/([^/?]+)'
    match = re.search(user_pattern, url)
    if match:
        sec_user_id = match.group(1)
        return CreatorUrlInfo(sec_user_id=sec_user_id)

    raise ValueError(f"Unable to parse creator ID from URL: {url}")


if __name__ == '__main__':
    # Test video URL parsing
    print("=== Video URL Parsing Test ===")
    test_urls = [
        "https://www.douyin.com/video/7525082444551310602",
        "https://www.douyin.com/user/MS4wLjABAAAATJPY7LAlaa5X-c8uNdWkvz0jUGgpw4eeXIwu_8BhvqE?from_tab_name=main&modal_id=7525082444551310602",
        "https://www.douyin.com/root/search/python?aid=b733a3b0-4662-4639-9a72-c2318fba9f3f&modal_id=7471165520058862848&type=general",
        "7525082444551310602",
    ]
    for url in test_urls:
        try:
            result = parse_video_info_from_url(url)
            print(f"✓ URL: {url[:80]}...")
            print(f"  Result: {result}\n")
        except Exception as e:
            print(f"✗ URL: {url}")

View on GitHub (pinned to d6f7c5bb90)

Solutions

  1. Pass the bare sec_user_id (the MS4wLjABAAAA... string) or the canonical https://www.douyin.com/user/<sec_user_id> URL.
  2. Pre-validate http(s) inputs with the /user/([^/?]+) regex before calling.
  3. Expand share/short links to their final form first.

Example fix

# before
parse_creator_info_from_url('https://www.douyin.com/search?author=x')

# after
parse_creator_info_from_url('https://www.douyin.com/user/MS4wLjABAAAATJPY7LAlaa5X-c8uNdWkvz0jUGgpw4eeXIwu_8BhvqE')
Defensive patterns

Strategy: validation

Validate before calling

import re
DY_USER_RE = re.compile(r'/user/([^/?]+)')
def extract_dy_sec_uid(u: str) -> str | None:
    u = u.strip()
    if u.startswith('MS4wLjABAAAA'):
        return u
    m = DY_USER_RE.search(u)
    return m.group(1) if m else None

Type guard

def is_parseable_dy_creator(u: str) -> bool:
    u = u.strip()
    return u.startswith('MS4wLjABAAAA') or bool(re.search(r'/user/[^/?]+', u))

Try / catch

try:
    info = parse_creator_info_from_url(url.strip())
except ValueError:
    logger.warning(f'not a douyin creator url/sec_uid: {url}')

Prevention

When it happens

Trigger: Passing a douyin.com URL whose creator path is not /user/... (e.g. a profile redirect through /profile/); an http(s) URL containing douyin.com but without a /user/ segment (search pages); a sec_user_id with a typo that still starts with MS4wLjABAAAA passes, but a truncated one wrapped in a full URL fails the regex.

Common situations: Copy-pasting the share page rather than the creator homepage; URL truncation losing the /user/ path; upstream layout changes on douyin.com.

Understand the failure class

Related errors

AI-assisted analysis of NanmiCoder/MediaCrawler@d6f7c5bb90 (2026-08-15). Data as JSON: /api/errors/075345d43bb0cb6a. Report an issue: GitHub.