python/cpython · error · AssertionError

unexpected char in internal subset (in %r)

Error message

unexpected char in internal subset (in %r)

What it means

AssertionError raised by _markupbase.ParserBase._parse_doctype_subset while scanning the internal subset of a DOCTYPE declaration: a '<' was found that is not followed by '!'. Inside an internal subset only markup declarations (<!ELEMENT, <!ATTLIST, <!ENTITY, <!NOTATION>, comments, and parameter-entity references) are legal, so anything else is fatal.

Source

Thrown at Lib/_markupbase.py:192

            self.handle_comment(rawdata[i+4: j])
        return match.end(0)

    # Internal -- scan past the internal subset in a <!DOCTYPE declaration,
    # returning the index just past any whitespace following the trailing ']'.
    def _parse_doctype_subset(self, i, declstartpos):
        rawdata = self.rawdata
        n = len(rawdata)
        j = i
        while j < n:
            c = rawdata[j]
            if c == "<":
                s = rawdata[j:j+2]
                if s == "<":
                    # end of buffer; incomplete
                    return -1
                if s != "<!":
                    self.updatepos(declstartpos, j + 1)
                    raise AssertionError(
                        "unexpected char in internal subset (in %r)" % s
                    )
                if (j + 2) == n:
                    # end of buffer; incomplete
                    return -1
                if (j + 4) > n:
                    # end of buffer; incomplete
                    return -1
                if rawdata[j:j+4] == "<!--":
                    j = self.parse_comment(j, report=0)
                    if j < 0:
                        return j
                    continue
                name, j = self._scan_name(j + 2, declstartpos)
                if j == -1:
                    return -1
                if name not in {"attlist", "element", "entity", "notation"}:
                    self.updatepos(declstartpos, j + 2)

View on GitHub (pinned to bc6749cc3b)

Solutions

  1. Fix the source: every '<' inside the internal subset must start a '<!' declaration or '<!--' comment.
  2. Remove the internal subset entirely if unused: replace '\[.*?\]' within the DOCTYPE before feeding.
  3. Catch AssertionError, use parser.getpos() to locate the line, and repair just that construct.
  4. For XML with real DTD subsets, switch to an XML parser (xml.etree, lxml) instead of html.parser.

Example fix

# before
HTMLParser().feed('<!DOCTYPE d [ <ELEMENT a (#PCDATA)> ]>')  # missing '!' -> AssertionError

# after
HTMLParser().feed('<!DOCTYPE d [ <!ELEMENT a (#PCDATA)> ]>')
# or drop the subset: HTMLParser().feed('<!DOCTYPE d>')
Defensive patterns

Strategy: try-catch

Validate before calling

import re

def internal_subset_ok(raw: str) -> bool:
    m = re.search(r'<!DOCTYPE[^\[]*\[([^\]]*)\]', raw, re.I | re.S)
    if not m:
        return True
    subset = m.group(1)
    return not re.search(r'<(?!!|\s*$)', subset)  # every '<' must start '<!'

Try / catch

try:
    parser.feed(data)
except AssertionError as e:
    if 'internal subset' in str(e):
        data = re.sub(r'(<!DOCTYPE[^\[]*)\[.*?\]', r'\1', data, flags=re.S | re.I)
        parser.reset(); parser.feed(data)
    else:
        raise

Prevention

When it happens

Trigger: HTMLParser.feed() on <!DOCTYPE r [ <element ... > ]> where a declaration is missing its '!', or stray '<' text inside the subset, e.g. <!DOCTYPE d [ < 5 ]>. The two-char slice s = rawdata[j:j+2] is echoed in the message.

Common situations: Hand-written DTD internal subsets with typos; HTML documents where a '<' comparison operator appears inside an (invalidly placed) doctype; chunked feeding that leaves a lone '<' at a buffer boundary inside a subset.

Related errors

AI-assisted analysis of python/cpython@bc6749cc3b (2026-08-14). Data as JSON: /api/errors/cee5f40755817a62. Report an issue: GitHub.