{"record":{"id":"b7eb1f933e76869b","repo":"abhigyanpatwari/GitNexus","slug":"circuit-open-tag-huggingface-download-circuit","errorCode":null,"errorMessage":"${CIRCUIT_OPEN_TAG}: HuggingFace download circuit is open after repeated network failures — will reset in ~${secsUntilReset}s","messagePattern":"(.+?): HuggingFace download circuit is open after repeated network failures — will reset in ~(.+?)s","errorType":"exception","errorClass":null,"httpStatus":null,"severity":"error","filePath":"gitnexus/src/core/embeddings/hf-env.ts","lineNumber":249,"sourceCode":"      ? Math.min(envTimeout, HF_MAX_TIMEOUT_MS)\n      : HF_DOWNLOAD_TIMEOUT_MS;\n  const resolvedMaxAttempts =\n    Number.isFinite(envMaxAttempts) && envMaxAttempts > 0\n      ? Math.min(Math.floor(envMaxAttempts), HF_MAX_ATTEMPTS_CAP)\n      : HF_MAX_ATTEMPTS;\n  const {\n    maxAttempts = resolvedMaxAttempts,\n    baseDelayMs = HF_BASE_DELAY_MS,\n    timeoutMs = resolvedTimeout,\n    circuit = hfDownloadCircuit,\n    onRetry,\n  } = options;\n  if (circuit.getState() === 'open') {\n    // Compute remaining cooldown without consuming a probe permit.\n    const openedAt = circuit.getOpenedAt();\n    const secsUntilReset =\n      openedAt !== null ? Math.ceil((circuit.getCooldownMs() - (Date.now() - openedAt)) / 1000) : 0;\n    throw new Error(\n      `${CIRCUIT_OPEN_TAG}: HuggingFace download circuit is open after repeated network failures` +\n        (secsUntilReset > 0 ? ` — will reset in ~${secsUntilReset}s` : ''),\n    );\n  }\n\n  // Retry budget delegated to `withRetry` from gitnexus-shared. The\n  // HF-specific bits — per-attempt timeout, network-vs-non-network\n  // classification, circuit-breaker recording, onRetry callback — wire\n  // through the `isRetryable` callback. `circuitTripped` is the\n  // sentinel that lets us replace the final thrown error with a\n  // CIRCUIT_OPEN_TAG message when the breaker tripped mid-loop.\n  let circuitTripped = false;\n\n  try {\n    return await withRetry(\n      async () => {\n        const result = await withDownloadTimeout(fn, timeoutMs);\n        circuit.recordSuccess();","sourceCodeStart":231,"sourceCodeEnd":267,"githubUrl":"https://github.com/abhigyanpatwari/GitNexus/blob/aac7515d2a8c50a1f8f923c6fb77218b333560d6/gitnexus/src/core/embeddings/hf-env.ts#L231-L267","documentation":"withHfDownloadRetry() checks the process-global HuggingFace download circuit breaker before starting any attempt. After 3 consecutive network failures (CB_FAILURE_THRESHOLD=3) the circuit opens for 60s (CB_RESET_TIMEOUT_MS); calling again during the cooldown fails fast with the seconds-until-reset instead of hammering an unreachable endpoint. This message is computed without consuming a probe permit.","triggerScenarios":"A model download already failed 3 times in this process (fetch failed / ECONNREFUSED / ENOTFOUND / ETIMEDOUT / ECONNRESET) and a new download attempt starts within the ~60s cooldown window — e.g. a retry loop around `analyze --embeddings` or an MCP server re-initializing the embedder.","commonSituations":"huggingface.co unreachable from corporate proxies, the GFW, or air-gapped networks; users or scripts that immediately rerun the failed command instead of waiting; flaky Wi-Fi where 3 attempts burn through quickly.","solutions":["Wait for the stated cooldown (~60s) before retrying.","Set HF_ENDPOINT to a reachable mirror, e.g. HF_ENDPOINT=https://hf-mirror.com npx gitnexus analyze --embeddings.","Fix the underlying network issue (proxy env vars HTTP(S)_PROXY, DNS, firewall) before retrying.","Pre-cache the model once on a machine with access (HF_HOME cache dir) and reuse it.","Switch to HTTP embedding mode (GITNEXUS_EMBEDDING_URL + GITNEXUS_EMBEDDING_MODEL) so no HF download is needed."],"exampleFix":"# before\nnpx gitnexus analyze --embeddings   # fails, retry instantly -> circuit open\nnpx gitnexus analyze --embeddings\n# after\nHF_ENDPOINT=https://hf-mirror.com npx gitnexus analyze --embeddings","handlingStrategy":"retry","validationCode":"// circuit state is internal; the practical pre-check is connectivity\nimport { isNetworkFetchError } from './hf-env'; // classify after first failure","typeGuard":"const isCircuitOpen = (e: unknown): boolean =>\n  e instanceof Error && e.message.includes('circuit is open');","tryCatchPattern":"try {\n  await getEmbedder();\n} catch (e) {\n  if (isCircuitOpen(e)) {\n    const secs = Number(/~(\\d+)s/.exec(e.message)?.[1] ?? 60);\n    await sleep((secs + 1) * 1000);\n    return getEmbedder(); // single bounded retry after cooldown\n  }\n  throw e;\n}","preventionTips":["Set HF_ENDPOINT to a proven-reachable mirror before the first download.","Cache the downloaded model (HF_HOME on shared/persistent storage) so the breaker never matters on later runs.","Do not tight-loop retries around embedding initialization; space them beyond the 60s cooldown."],"tags":["embeddings","huggingface","circuit-breaker","network","rate-limiting"],"backgroundTag":"circuit-breaker-open","analyzedSha":"aac7515d2a8c50a1f8f923c6fb77218b333560d6","analyzedAt":"2026-08-20T23:29:22.980Z","contentChangedAt":"2026-08-20T23:29:22.980Z","schemaVersion":2},"datasetVersion":"2026-09-14T00:17:10.932Z"}