{"record":{"id":"b47141fdfed01e67","repo":"jeecgboot/JeecgBoot","slug":"url-b47141","errorCode":null,"errorMessage":"网页内容为空，请检查URL是否可访问","messagePattern":"网页内容为空，请检查URL是否可访问","errorType":"exception","errorClass":"JeecgBootException","httpStatus":null,"severity":"warning","filePath":"jeecg-boot/jeecg-boot-module/jeecg-boot-module-airag/src/main/java/org/jeecg/modules/airag/llm/handler/EmbeddingHandler.java","lineNumber":780,"sourceCode":"     * @date 2026/3/19\n     */\n    private String parseWebPage(AiragKnowledgeDoc doc) {\n        String metadata = doc.getMetadata();\n        AssertUtils.assertNotEmpty(\"请先配置网页URL\", metadata);\n        JSONObject metadataJson = JSONObject.parseObject(metadata);\n        String website = metadataJson.getString(LLMConsts.KNOWLEDGE_DOC_METADATA_WEBSITE);\n        AssertUtils.assertNotEmpty(\"请先配置网页URL\", website);\n\n        Matcher matcher = LLMConsts.WEB_PATTERN.matcher(website);\n        if (!matcher.matches()) {\n            throw new JeecgBootException(\"网页URL格式不正确，请以http://或https://开头\");\n        }\n\n        try {\n            WebPageParser webPageParser = new WebPageParser();\n            String content = webPageParser.parseToMarkdown(website);\n            if (oConvertUtils.isEmpty(content)) {\n                throw new JeecgBootException(\"网页内容为空，请检查URL是否可访问\");\n            }\n            log.info(\"网页解析成功, URL: {}, 内容长度: {}\", website, content.length());\n            return content;\n        } catch (JeecgBootException e) {\n            throw e;\n        } catch (Exception e) {\n            log.error(\"网页解析失败, URL: {}, 错误: {}\", website, e.getMessage(), e);\n            throw new JeecgBootException(\"网页解析失败: \" + e.getMessage());\n        }\n    }\n\n    /**\n     * 解析文件\n     *\n     * @param doc\n     * @author chenrui\n     * @date 2025/3/5 11:31\n     */","sourceCodeStart":762,"sourceCodeEnd":798,"githubUrl":"https://github.com/jeecgboot/JeecgBoot/blob/96fb33f5ec68516da0b0147da06b2eb0419e063a/jeecg-boot/jeecg-boot-module/jeecg-boot-module-airag/src/main/java/org/jeecg/modules/airag/llm/handler/EmbeddingHandler.java#L762-L798","documentation":"Thrown by EmbeddingHandler.parseWebPage() when WebPageParser.parseToMarkdown(website) returns an empty or null result. This means the HTTP fetch succeeded (or returned a response) but the extracted content was empty — the page exists but yielded no parseable text.","triggerScenarios":"The target URL is reachable but returns a page with no extractable text content: a JavaScript-only SPA with no server-rendered HTML; a page that returned an empty body; a page blocked by a CAPTCHA or paywall returning minimal HTML; a page that returned a 200 status with an error/landing page body.","commonSituations":"Target site is a single-page application whose content loads via JavaScript (WebPageParser likely does not execute JS); the URL redirects to a login/error page with no content; the site blocks scraping bots and returns an empty response; network proxy interferes and returns an empty body.","solutions":["Open the URL in a browser to confirm it has visible text content (not purely JavaScript-rendered).","If the page requires JavaScript, consider pre-rendering the page or using a different source.","Check that the URL is not behind authentication or a paywall.","Verify network connectivity and proxy settings allow the server to reach the target URL."],"exampleFix":null,"handlingStrategy":"try-catch","validationCode":null,"typeGuard":null,"tryCatchPattern":"try {\n    String content = parseWebPage(doc);\n} catch (JeecgBootException e) {\n    if (e.getMessage().contains(\"网页内容为空\")) {\n        // suggest user verify the URL renders server-side content\n        throw new JeecgBootException(\"网页内容为空，该页面可能需要JavaScript渲染，请尝试其他URL\");\n    }\n    throw e;\n}","preventionTips":["Verify target URLs return server-rendered HTML content (not JS-only SPAs).","Test URLs in a browser with JavaScript disabled to check for server-rendered content.","Avoid URLs behind authentication, paywalls, or anti-bot protections."],"tags":["web-parsing","content-extraction","rag"],"backgroundTag":null,"analyzedSha":"96fb33f5ec68516da0b0147da06b2eb0419e063a","analyzedAt":"2026-08-14T00:04:16.786Z","schemaVersion":2},"datasetVersion":"2026-08-14T00:17:13.853Z"}