scrapy/scrapy · error · ValueError

No <form> element found in {response}

Error message

No <form> element found in {response}

What it means

_get_form() raises ValueError when the response contains no <form> element at all (root.xpath('//form') is empty). It backs FormRequest.from_response(); before any formname/formid/formxpath/formnumber matching can happen, at least one form must exist, so a form-less page (JS-rendered form, wrong URL, login moved) fails immediately.

Source

Thrown at scrapy/http/request/form.py:184

        (to_bytes(k, enc), to_bytes(v, enc))
        for k, vs in seq
        for v in (vs if is_listlike(vs) else [cast("str", vs)])
    ]
    return urlencode(values, doseq=True)


def _get_form(
    response: TextResponse,
    formname: str | None,
    formid: str | None,
    formnumber: int,
    formxpath: str | None,
) -> FormElement:
    """Find the wanted form element within the given response."""
    root = response.selector.root
    forms = root.xpath("//form")
    if not forms:
        raise ValueError(f"No <form> element found in {response}")

    if formname is not None:
        f = root.xpath(f'//form[@name="{formname}"]')
        if f:
            return cast("FormElement", f[0])

    if formid is not None:
        f = root.xpath(f'//form[@id="{formid}"]')
        if f:
            return cast("FormElement", f[0])

    # Get form element from xpath, if not found, go up
    if formxpath is not None:
        nodes = root.xpath(formxpath)
        if nodes:
            el = nodes[0]
            while True:
                if el.tag == "form":

View on GitHub (pinned to 06af687662)

Solutions

1. Inspect the response first: log response.css('form').get() and response.url to confirm what actually came back
2. If the form is JS-built, use scrapy-playwright or splash to render, or reverse-engineer the underlying POST endpoint and send a plain FormRequest directly
3. Handle redirects/logins: check response.status and follow to the real form page before calling from_response
4. Catch ValueError around from_response as a guard so one bad page does not kill the spider

Example fix

# before
yield FormRequest.from_response(response, formdata={'user': 'u', 'pass': 'p'})
# page has no <form> -> ValueError

# after
if response.css('form'):
    yield FormRequest.from_response(response, formdata={'user': 'u', 'pass': 'p'})
else:
    self.logger.warning('No form at %s', response.url)

def has_form(response) -> bool:
    return bool(response.css('form'))

if has_form(response):
    yield FormRequest.from_response(response, formdata={...})
else:
    self.logger.warning('no form on %s', response.url)

null

try:
    req = FormRequest.from_response(response, formdata=data)
except ValueError as e:
    self.logger.warning('form extraction failed on %s: %s', response.url, e)
    return

When it happens

Trigger: FormRequest.from_response(response) on a page whose form is injected by JavaScript (plain Scrapy does not execute JS); response is an error/login-redirect page with no forms; parsing a page whose forms live inside an <iframe> document (not in the main response body); Content-Type or encoding issues leaving the body unparsed as expected.

Common situations: Login endpoints changed or now require JS; the target page redirected to a captcha/consent page; site moved form into a frame loaded by a second request; response.follow to the wrong URL due to relative-link mistakes.

Related errors

• No <form> element found with {formxpath}
• Form number {formnumber} not found in {response}
• formdata should be a dict or iterable of tuples
• Multiple elements found ({el!r}) matching the criteria in cl
• No clickable element matching clickdata: {clickdata!r}

AI-assisted analysis of scrapy/scrapy@06af687662 (2026-08-15). Data as JSON: /api/errors/263f6df5e6eab765. Report an issue: GitHub.