python/cpython · error · AssertionError

unexpected char %r in internal subset

Error message

unexpected char %r in internal subset

What it means

AssertionError raised by _markupbase.ParserBase._parse_doctype_subset when a character inside the DOCTYPE internal subset is not '<', '%', ']', or whitespace. It is the subset scanner's catch-all for stray content (text, quotes, operators) that is illegal between declarations, echoing the offending character.

Source

Thrown at Lib/_markupbase.py:244

                    return j
                if rawdata[j] == ";":
                    j = j + 1
            elif c == "]":
                j = j + 1
                while j < n and rawdata[j].isspace():
                    j = j + 1
                if j < n:
                    if rawdata[j] == ">":
                        return j
                    self.updatepos(declstartpos, j)
                    raise AssertionError("unexpected char after internal subset")
                else:
                    return -1
            elif c.isspace():
                j = j + 1
            else:
                self.updatepos(declstartpos, j)
                raise AssertionError("unexpected char %r in internal subset" % c)
        # end of buffer reached
        return -1

    # Internal -- scan past <!ELEMENT declarations
    def _parse_doctype_element(self, i, declstartpos):
        name, j = self._scan_name(i, declstartpos)
        if j == -1:
            return -1
        # style content model; just skip until '>'
        rawdata = self.rawdata
        if '>' in rawdata[j:]:
            return rawdata.find(">", j) + 1
        return -1

    # Internal -- scan past <!ATTLIST declarations
    def _parse_doctype_attlist(self, i, declstartpos):
        rawdata = self.rawdata
        name, j = self._scan_name(i, declstartpos)

View on GitHub (pinned to bc6749cc3b)

Solutions

  1. Keep the internal subset to declarations only; move text outside the DOCTYPE.
  2. Balance and terminate entity values (<!ENTITY e "v">) so quotes do not leak past ']'.
  3. Catch AssertionError, use getpos() to locate the character, and repair or strip the subset (re.sub(r'\[[^\]]*\]', '', doctype)).
  4. Drop the subset entirely when it is not semantically needed: feed '<!DOCTYPE root>' instead.

Example fix

# before
HTMLParser().feed('<!DOCTYPE d [ version 1.0 ]>')  # AssertionError: unexpected char

# after
HTMLParser().feed('<!DOCTYPE d>')
# or keep only legal content: '<!DOCTYPE d [ <!ENTITY ver "1.0"> ]>'
Defensive patterns

Strategy: try-catch

Validate before calling

import re

def subset_content_ok(raw: str) -> bool:
    m = re.search(r'<!DOCTYPE[^\[]*\[([^\]]*)\]', raw, re.I | re.S)
    if not m:
        return True
    body = re.sub(r'<!--.*?-->|<!\w+[^>]*>|%[^;]*;|\s+', '', m.group(1), flags=re.S)
    return body == ''  # nothing but decls, comments, PE refs, whitespace

Try / catch

try:
    parser.feed(data)
except AssertionError as e:
    if 'in internal subset' in str(e):
        data = re.sub(r'(<!DOCTYPE[^\[]*)\[.*?\]', r'\1', data, flags=re.S | re.I)
        parser.reset(); parser.feed(data)
    else:
        raise

Prevention

When it happens

Trigger: HTMLParser.feed() on <!DOCTYPE d [ stray text ]>, <!DOCTYPE d [ "unterminated ]>, or any subset containing free text or stray punctuation outside of a proper <!...> declaration, %-reference, or the closing ']'.

Common situations: Documents with prose accidentally inside the doctype (bad template slots); quote-unbalanced entity values swallowing the ']'; feeding binary/mojibake data; hand-crafted test fixtures with sloppy subsets.

Related errors

AI-assisted analysis of python/cpython@bc6749cc3b (2026-08-14). Data as JSON: /api/errors/db41e1c0496ff465. Report an issue: GitHub.