{"record":{"id":"e5ef5fd5f236d35a","repo":"Textualize/textual","slug":"invalid-utf-8-byte-first-byte","errorCode":null,"errorMessage":"Invalid UTF-8 byte: {first_byte}","messagePattern":"Invalid UTF-8 byte: (.+?)","errorType":"exception","errorClass":"ValueError","httpStatus":null,"severity":"error","filePath":"src/textual/widgets/_text_area.py","lineNumber":2792,"sourceCode":"\n    while current_byte_offset < len(data):\n        byte_to_codepoint[current_byte_offset] = code_point_offset\n        first_byte = data[current_byte_offset]\n\n        # Single-byte character\n        if (first_byte & 0b10000000) == 0:\n            current_byte_offset += 1\n        # 2-byte character\n        elif (first_byte & 0b11100000) == 0b11000000:\n            current_byte_offset += 2\n        # 3-byte character\n        elif (first_byte & 0b11110000) == 0b11100000:\n            current_byte_offset += 3\n        # 4-byte character\n        elif (first_byte & 0b11111000) == 0b11110000:\n            current_byte_offset += 4\n        else:\n            raise ValueError(f\"Invalid UTF-8 byte: {first_byte}\")\n\n        code_point_offset += 1\n\n    # Mapping for the end of the string\n    byte_to_codepoint[current_byte_offset] = code_point_offset\n    return byte_to_codepoint\n","sourceCodeStart":2774,"sourceCodeEnd":2799,"githubUrl":"https://github.com/Textualize/textual/blob/06dbeef4bb70fb718236aa418ed658ef4667a126/src/textual/widgets/_text_area.py#L2774-L2799","documentation":"Raised while Textual's TextArea builds a byte-offset-to-codepoint map: the first byte of a character matches none of the UTF-8 leading-byte patterns (0xxxxxxx, 110xxxxx, 1110xxxx, 11110xxx). This means the input string's underlying bytes are not valid UTF-8, so offsets cannot be computed.","triggerScenarios":"Calling TextArea/document APIs that index locations (e.g. setting text, moving the cursor, or computing offsets) with a `str` whose bytes contain invalid UTF-8 sequences — typically text decoded with error-tolerant codecs (surrogateescape) or constructed from raw bytes containing values like 0xFF/0xFE as leading bytes.","commonSituations":"Loading files opened in binary and decoded with errors='replace'/'surrogateescape', embedding literal bytes in source, or receiving data from sockets/subprocess output that isn't UTF-8 (e.g. latin-1 or CP1252 logs).","solutions":["Sanitize text before assigning to TextArea: `text.encode('utf-8', errors='ignore').decode('utf-8')` or use `errors='replace'`.","Decode source data explicitly with the correct codec (e.g. `data.decode('cp1252')`) instead of assuming UTF-8.","If raw bytes must be preserved, display a hex view or repr() rather than passing them as text."],"exampleFix":"# before\ntext_area.text = raw_bytes.decode('utf-8', errors='surrogateescape')\n# after\ntext_area.text = raw_bytes.decode('utf-8', errors='replace')","handlingStrategy":"validation","validationCode":"def safe_text(raw: bytes | str) -> str:\n    s = raw.decode('utf-8', errors='replace') if isinstance(raw, bytes) else raw\n    s.encode('utf-8')  # raises if still invalid\n    return s\ntext_area.text = safe_text(data)","typeGuard":"def is_valid_utf8_text(s: str) -> bool:\n    try:\n        s.encode('utf-8')\n        return True\n    except UnicodeEncodeError:\n        return False","tryCatchPattern":"try:\n    text_area.text = value\nexcept ValueError:\n    text_area.text = value.encode('utf-8', 'replace').decode('utf-8')","preventionTips":["Always decode external bytes with an explicit codec and errors='replace'","Never pass surrogateescape-decoded strings to Textual widgets"],"tags":["utf-8","encoding","text-area","textual","validation"],"backgroundTag":"invalid-utf-8-decoding","analyzedSha":"06dbeef4bb70fb718236aa418ed658ef4667a126","analyzedAt":"2026-08-27T02:36:57.214Z","schemaVersion":2},"datasetVersion":"2026-08-27T03:17:27.898Z"}