{"record":{"id":"e3000ff4fc82f7c3","repo":"alibaba/spring-ai-alibaba","slug":"unsupported-charset-falling-back-to-utf-8","errorCode":null,"errorMessage":"Unsupported charset '{}', falling back to UTF-8","messagePattern":"Unsupported charset '(.+?)', falling back to UTF-8","errorType":"console","errorClass":null,"httpStatus":null,"severity":"info","filePath":"spring-ai-alibaba-agent-framework/src/main/java/com/alibaba/cloud/ai/graph/agent/tools/WebFetchTool.java","lineNumber":361,"sourceCode":"\t\t}\n\t\tcatch (InterruptedException e) {\n\t\t\tThread.currentThread().interrupt();\n\t\t\tthrow new WebFetchException(\"Request was interrupted\", e);\n\t\t}\n\t}\n\n\tprivate Optional<Charset> extractCharset(HttpResponse<?> response) {\n\t\treturn response.headers()\n\t\t\t.firstValue(\"Content-Type\")\n\t\t\t.flatMap(contentType -> {\n\t\t\t\tMatcher matcher = CHARSET_PATTERN.matcher(contentType);\n\t\t\t\tif (matcher.find()) {\n\t\t\t\t\tString charsetName = matcher.group(1);\n\t\t\t\t\ttry {\n\t\t\t\t\t\treturn Optional.of(Charset.forName(charsetName));\n\t\t\t\t\t}\n\t\t\t\t\tcatch (Exception e) {\n\t\t\t\t\t\tlogger.warn(\"Unsupported charset '{}', falling back to UTF-8\", charsetName);\n\t\t\t\t\t\treturn Optional.empty();\n\t\t\t\t\t}\n\t\t\t\t}\n\t\t\t\treturn Optional.empty();\n\t\t\t});\n\t}\n\n\tprivate String summarize(String content, String userQuery) {\n\t\ttry {\n\t\t\tString response = this.chatClient.prompt()\n\t\t\t\t.user(u -> u.text(FETCH_SUMMARIZE_PROMPT).param(\"content\", content).param(\"userQuery\", userQuery))\n\t\t\t\t.call()\n\t\t\t\t.content();\n\t\t\treturn response != null ? response : \"Error: Received empty response from AI model\";\n\t\t}\n\t\tcatch (Exception e) {\n\t\t\tlogger.error(\"Failed to summarize content\", e);\n\t\t\treturn \"Error summarizing content: \" + e.getMessage();","sourceCodeStart":343,"sourceCodeEnd":379,"githubUrl":"https://github.com/alibaba/spring-ai-alibaba/blob/f82da0b50f35744c13968191be2b1cd2452ef550/spring-ai-alibaba-agent-framework/src/main/java/com/alibaba/cloud/ai/graph/agent/tools/WebFetchTool.java#L343-L379","documentation":"extractCharset found a charset name in the Content-Type header or HTML meta tag that Charset.forName does not recognize on this JVM; it logs the warning and returns empty, causing the tool to fall back to decoding the body as UTF-8.","triggerScenarios":"A page declares a misspelled, non-standard, or platform-specific charset (e.g. 'utf8' variants, 'gb2312' aliases absent from the charset provider, garbage values) in its metadata.","commonSituations":"Scraping legacy Chinese/European sites declaring charsets like 'GB2312' or 'ISO-8859-1' aliases on restricted JVMs, pages with malformed meta tags, CDN-injected wrong headers.","solutions":["Usually safe to ignore: content decodes as UTF-8; check output for mojibake.","If decoding is wrong, pre-normalize the source page or serve it with a correct Content-Type header.","Ensure the JVM has the full charsets module (java.base plus jdk.charsets) if exotic encodings are needed.","Fix the upstream page's charset declaration if you control it."],"exampleFix":null,"handlingStrategy":"fallback","validationCode":"try { Charset.forName(declared); } catch (Exception e) { /* declared charset unsupported; expect UTF-8 fallback */ }","typeGuard":null,"tryCatchPattern":null,"preventionTips":["Serve pages with a correct, standard charset in Content-Type.","Test scraping targets for mojibake after the UTF-8 fallback.","Run on a JVM with the full charset provider if exotic encodings are expected."],"tags":["charset","encoding","webfetch","fallback"],"backgroundTag":"invalid-argument-value","analyzedSha":"f82da0b50f35744c13968191be2b1cd2452ef550","analyzedAt":"2026-09-09T15:32:42.421Z","contentChangedAt":"2026-09-09T15:32:42.421Z","schemaVersion":2},"datasetVersion":"2026-09-17T15:17:12.973Z"}