{"record":{"id":"51815f5e161b42a6","repo":"sigoden/aichat","slug":"invalid-gh-tree","errorCode":null,"errorMessage":"Invalid gh tree {}","messagePattern":"Invalid gh tree (.+?)","errorType":"exception","errorClass":"anyhow::Error","httpStatus":null,"severity":"error","filePath":"src/utils/request.rs","lineNumber":308,"sourceCode":"        }\n        paths.extend(new_paths);\n\n        index += batch.len();\n    }\n\n    Ok(result_pages)\n}\n\n#[derive(Debug, Deserialize)]\npub struct Page {\n    pub path: String,\n    pub text: String,\n}\n\nasync fn crawl_gh_tree(start_url: &Url, exclude: &[String]) -> Result<Vec<String>> {\n    let path_segs: Vec<&str> = start_url.path().split('/').collect();\n    if path_segs.len() < 4 {\n        bail!(\"Invalid gh tree {}\", start_url.as_str());\n    }\n    let client = match *CLIENT {\n        Ok(ref client) => client,\n        Err(ref err) => bail!(\"{err}\"),\n    };\n    let owner = path_segs[1];\n    let repo = path_segs[2];\n    let branch = path_segs[4];\n    let root_path = path_segs[5..].join(\"/\");\n\n    let url = format!(\"https://api.github.com/repos/{owner}/{repo}/git/ref/heads/{branch}\");\n\n    let res_body: Value = client\n        .get(&url)\n        .header(\"User-Agent\", USER_AGENT)\n        .header(\"Accept\", \"application/vnd.github+json\")\n        .header(\"X-GitHub-Api-Version\", \"2022-11-28\")\n        .send()","sourceCodeStart":290,"sourceCodeEnd":326,"githubUrl":"https://github.com/sigoden/aichat/blob/82976d349ad97ac9aae0655ad631dace5e2a6385/src/utils/request.rs#L290-L326","documentation":"crawl_gh_tree parses a GitHub tree/blob URL into path segments to extract owner, repo, branch, and root path. It requires at least 4 segments in the URL path (origin/host + owner/repo/...); URLs that don't match a GitHub repository layout (e.g. a bare github.com page or a short URL) are rejected with 'Invalid gh tree {url}'.","triggerScenarios":"Calling crawl_website with a start_url whose path has fewer than 4 segments, e.g. https://github.com or https://github.com/owner — not a full repo tree/blob URL like https://github.com/owner/repo/tree/branch/dir.","commonSituations":"Passing a GitHub repository root or user profile URL instead of a tree/blob URL; a typo dropping the repo or branch segment; expecting generic web crawling on a GitHub URL that only supports tree crawling.","solutions":["Pass a full GitHub tree or blob URL with owner, repo, and branch, e.g. https://github.com/owner/repo/tree/main","Check the URL for missing segments (repo name or branch omitted)","If you want to crawl general GitHub pages, use a non-GitHub crawl path or the HTML crawl flow instead of the gh tree mode"],"exampleFix":"// before\ncrawl_website(\"https://github.com/sigoden\", options)?;\n// after\ncrawl_website(\"https://github.com/sigoden/aichat/tree/main\", options)?;","handlingStrategy":"validation","validationCode":"fn is_valid_gh_tree_url(u: &str) -> bool {\n    url::Url::parse(u).map(|u| {\n        u.host_str().map_or(false, |h| h.contains(\"github.com\"))\n            && u.path().split('/').filter(|s| !s.is_empty()).count() >= 3\n    }).unwrap_or(false)\n}\n// if !is_valid_gh_tree_url(start_url) { /* fix URL before crawling */ }","typeGuard":null,"tryCatchPattern":"match crawl_website(start_url, options).await {\n    Err(e) if e.to_string().starts_with(\"Invalid gh tree\") => {\n        eprintln!(\"{e}: provide a full github tree/blob URL like https://github.com/owner/repo/tree/branch\");\n    }\n    other => other?,\n}","preventionTips":["Always pass URLs of the form https://github.com/owner/repo/tree/branch[/path]","Validate URL segment count before invoking gh-tree crawling","Use the generic HTML crawl flow for non-tree GitHub pages"],"tags":["url","github","crawler","format"],"backgroundTag":"invalid-url-format","analyzedSha":"82976d349ad97ac9aae0655ad631dace5e2a6385","analyzedAt":"2026-09-09T18:33:06.139Z","contentChangedAt":"2026-09-09T18:33:06.139Z","schemaVersion":2},"datasetVersion":"2026-09-16T09:17:16.951Z"}