midudev/autoskills · error · Error

Tarball fetch failed

Error message

Tarball fetch failed: ${res.status} ${url}

What it means

downloadTarball fetches a GitHub codeload tarball URL over HTTP and streams it to a file. If the response is not ok (non-2xx) or has no body, it throws this error including the HTTP status and URL. Common statuses: 404 (repo/branch/tag gone), 403 (rate limit), 5xx (GitHub server error).

Solutions

  1. Re-run after checking the URL in a browser — if 404, the ref/repo no longer exists
  2. Set a valid GITHUB_TOKEN to raise the API/codeload rate limit; if 403, wait for the rate-limit reset
  3. Check https://www.githubstatus.com for GitHub outages on 5xx
  4. If status is from an abort (timeout), raise TARBALL_TIMEOUT_MS for large repos

Example fix

// before
// no token -> 403 rate limited
await downloadTarball(url, dest);
// after
process.env.GITHUB_TOKEN = "ghp_..."; // authenticated requests
await downloadTarball(url, dest);
Defensive patterns

Strategy: retry

Validate before calling

// pre-check the tarball URL responds 200
const probe = await fetch(url, { method: "HEAD", headers: GITHUB_TOKEN ? { Authorization: `Bearer ${GITHUB_TOKEN}` } : {} });
if (!probe.ok) throw new Error(`tarball URL not fetchable: ${probe.status} ${url}`);

Try / catch

async function downloadWithRetry(url, dest, attempts = 3) {
  for (let i = 0; i < attempts; i++) {
    try { return await downloadTarball(url, dest); }
    catch (err) {
      if (!err.message.startsWith("Tarball fetch failed")) throw err;
      const status = parseInt(err.message.match(/(\d{3})/)?.[1] ?? "0", 10);
      if (status === 404) throw err; // permanent
      await new Promise(r => setTimeout(r, 2 ** i * 1000));
    }
  }
  throw new Error(`tarball download failed after ${attempts} attempts`);
}

Prevention

When it happens

Trigger: fetch(url) returns res.ok === false or res.body === null while downloading a repo tarball — 404 for missing ref, 403 rate-limited without/with an exhausted GITHUB_TOKEN, 5xx from GitHub.

Common situations: Hitting GitHub's unauthenticated rate limit during bulk syncs; syncing a repo branch/tag that was deleted; transient GitHub 5xx; GITHUB_TOKEN expired or revoked.

Understand the failure class

Background: "API error: {status}" and "HTTP 401/403/404/429/5xx" errors: non-2xx HTTP responses explained — this error's family across 27 libraries.

Related errors

AI-assisted analysis of midudev/autoskills@0ec725320d (2026-09-15). Data as JSON: /api/errors/998da52f7d67b6b3. Report an issue: GitHub.

Appendix: source

Thrown at packages/autoskills/scripts/sync-skills.mjs:220

    sizeKB: body.size || 0,
  };
}

// Tarball download size threshold (KB). Above this we use per-file fetch.
const HEAVY_REPO_KB = 50_000;
// Hard timeout for tarball downloads. Some repos have slow codeload CDNs.
const TARBALL_TIMEOUT_MS = 180_000;

async function downloadTarball(repo, sha, destFile) {
  const url = `https://codeload.github.com/${repo}/tar.gz/${sha}`;
  const headers = { "User-Agent": "autoskills-sync" };
  if (GITHUB_TOKEN) headers.Authorization = `Bearer ${GITHUB_TOKEN}`;
  const ac = new AbortController();
  const timer = setTimeout(() => ac.abort(), TARBALL_TIMEOUT_MS);
  try {
    const res = await fetch(url, { headers, signal: ac.signal });
    if (!res.ok || !res.body) {
      throw new Error(`Tarball fetch failed: ${res.status} ${url}`);
    }
    await pipeline(res.body, createWriteStream(destFile));
  } finally {
    clearTimeout(timer);
  }
}

async function fetchRepoTree(repo, sha) {
  const res = await ghFetch(`https://api.github.com/repos/${repo}/git/trees/${sha}?recursive=1`);
  const body = await res.json();
  if (body.truncated) {
    throw new Error(`git tree truncated for ${repo}@${sha.slice(0, 7)}`);
  }
  return body.tree || [];
}

function findSkillDirsInTree(tree, skillName) {
  // Returns array of { dir } where dir/SKILL.md exists in the tree.

View on GitHub (pinned to 0ec725320d)