sigoden/aichat · error · anyhow::Error

Invalid github repo tree

Error message

Invalid github repo tree

What it means

After fetching `/repos/{owner}/{repo}/git/trees/{sha}?recursive=true`, `crawl_gh_tree` expects the JSON body to contain a `tree` array and returns `anyhow!("Invalid github repo tree")` when `res_body["tree"].as_array()` is None. This indicates the API replied with an unexpected payload shape — usually a GitHub error object (message/errors fields) rather than a tree object.

Solutions

1. Inspect the raw response body and HTTP status — handle GitHub error messages and rate limits explicitly before parsing.
2. For huge repos, avoid `recursive=true`: fetch the tree level by level or use the git archive/tarball endpoint.
3. Re-check authentication: adding a token raises rate limits and unlocks private repos.
4. Re-fetch the SHA (see 'Not found branch or tag' path) if the ref may have moved.

Example fix

// before
let res_body: Value = client.get(&url).send().await?.json().await?;
let tree = res_body["tree"].as_array()
    .ok_or_else(|| anyhow!("Invalid github repo tree"))?;
// after
let resp = client.get(&url).send().await?;
let res_body: Value = resp.json().await?;
if let Some(msg) = res_body.get("message").and_then(|m| m.as_str()) {
    anyhow::bail!("GitHub API error: {msg}");
}
let tree = res_body["tree"].as_array()
    .ok_or_else(|| anyhow!("Invalid github repo tree"))?;

// pre-flight: confirm the repo is small enough for a recursive tree
async fn tree_fits(owner: &str, repo: &str) -> bool {
    // GitHub truncates/errors on very large recursive trees; check repo size (KB)
    let body: serde_json::Value = reqwest::get(format!("https://api.github.com/repos/{owner}/{repo}"))
        .await.ok()?.json().await.ok()?;
    body["size"].as_i64().map(|s| s < 500_000).unwrap_or(false)
}

for attempt in 0..3 {
    match crawl_gh_tree(owner, repo, r, &client).await {
        Err(e) if e.to_string() == "Invalid github repo tree" && attempt < 2 => {
            tokio::time::sleep(Duration::from_secs(2u64.pow(attempt))).await; // likely rate limit
        }
        other => { other?; break; }
    }
}

When it happens

Trigger: The trees API returns an error/oversized-response body instead of a tree: too many files for `recursive=true` (truncated or rejected), wrong SHA, rate-limited response, or a malformed/non-JSON payload.

Common situations: Crawling a very large monorepo where the recursive tree request is rejected or truncated; hitting GitHub's unauthenticated rate limit mid-crawl; an invalid or stale SHA passed to the trees endpoint.

Related errors

• Not found branch or tag
• Invalid gh tree
• Invalid response data
• The call ' ' has invalid arguments
• No valid models

AI-assisted analysis of sigoden/aichat@82976d349a (2026-09-09). Data as JSON: /api/errors/44b3f7e6e9cbc9af. Report an issue: GitHub.

Appendix: source

Thrown at src/utils/request.rs:348

    let sha = res_body["object"]["sha"]
        .as_str()
        .ok_or_else(|| anyhow!("Not found branch or tag"))?;

    let url = format!("https://api.github.com/repos/{owner}/{repo}/git/trees/{sha}?recursive=true");

    let res_body: Value = client
        .get(&url)
        .header("User-Agent", USER_AGENT)
        .header("Accept", "application/vnd.github+json")
        .header("X-GitHub-Api-Version", "2022-11-28")
        .send()
        .await?
        .json()
        .await?;
    let tree = res_body["tree"]
        .as_array()
        .ok_or_else(|| anyhow!("Invalid github repo tree"))?;
    let paths = tree
        .iter()
        .flat_map(|v| {
            let typ = v["type"].as_str()?;
            let path = v["path"].as_str()?;
            if typ == "blob"
                && (path.ends_with(".md") || path.ends_with(".MD"))
                && path.starts_with(&root_path)
                && !should_exclude_link(path, exclude)
            {
                Some(format!(
                    "https://raw.githubusercontent.com/{owner}/{repo}/{branch}/{path}"
                ))
            } else {
                None
            }
        })
        .collect();

View on GitHub (pinned to 82976d349a)