KeygraphHQ/shannon · error · PentestError

GIT_CHECKPOINT_FAILED

GIT_CHECKPOINT_FAILED

Error message

Git command failed after ${maxRetries} retries

What it means

Thrown by executeGitCommandWithRetry when a git invocation (checkpoint commit, reset, clean, rev-list) cannot complete within the retry budget. The function retries only on git lock contention (index.lock, 'unable to lock', 'Another git process', etc.) using exponential backoff (2^(attempt-1) * 1000ms, default 5 attempts); any non-lock git error is re-thrown immediately. The PentestError is marked retryable=true with code GIT_CHECKPOINT_FAILED. Note: on the final retry attempt the raw underlying exec error is actually re-thrown at the `throw error` line, so this specific PentestError message is a defensive terminal guard for the loop-exit path.

Source

Thrown at apps/worker/src/services/git-manager.ts:227

      return result;
    } catch (error) {
      const errMsg = error instanceof Error ? error.message : String(error);

      if (isGitLockError(errMsg) && attempt < maxRetries) {
        const delay = 2 ** (attempt - 1) * 1000;
        // executeGitCommandWithRetry is also called outside activity context
        // (e.g., from resume logic), so we use console.warn as a fallback here
        console.warn(
          `Git lock conflict during ${description} (attempt ${attempt}/${maxRetries}). Retrying in ${delay}ms...`,
        );
        await new Promise((resolve) => setTimeout(resolve, delay));
        continue;
      }

      throw error;
    }
  }
  throw new PentestError(
    `Git command failed after ${maxRetries} retries`,
    'filesystem',
    true, // Retryable - transient git lock issues
    { maxRetries, description },
    ErrorCode.GIT_CHECKPOINT_FAILED,
  );
}

// Two-phase reset: hard reset (tracked files) + clean (untracked files).
// When paths is provided, the untracked clean is scoped to those paths so a
// failing agent's rollback can't delete a concurrent sibling agent's scratch.
export async function rollbackGitWorkspace(
  sourceDir: string,
  reason: string = 'retry preparation',
  logger: ActivityLogger,
  paths?: readonly string[],
): Promise<GitOperationResult> {
  // Skip git operations if not a git repository

View on GitHub (pinned to 1ae0a142f8)

Solutions

1. Inspect the workspace deliverables .git dir for a stale index.lock and remove it: rm -f <repo>/.git/index.lock (and any *.lock), then resume.
2. Ensure only one scan targets a given workspace/repo at a time; parallel agents already share an in-process lock but two CLI invocations do not.
3. Increase maxRetries where the call site accepts it, or widen the backoff window if the backing filesystem is known-slow.
4. Re-run from a clean workspace (./shannon stop --clean) if the git repo state is irrecoverably corrupted.
5. Check container mount performance; move the repo off a network/overlay filesystem onto local disk.

Example fix

// before: stale lock blocks all retries
//   ls <repo>/.git/index.lock  -> file exists from a crashed worker
// after: clear the stale lock, then resume
//   rm -f <repo>/.git/index.lock
//   ./shannon start -u <url> -r <repo> -w <same-workspace>

// Before checkpointing, confirm no stale lock and that the repo is a git repo
import { existsSync } from 'node:fs';
const lockPath = path.join(repoPath, '.git', 'index.lock');
if (existsSync(lockPath)) {
  const stat = await fs.stat(lockPath);
  const ageMs = Date.now() - stat.mtimeMs;
  if (ageMs > 60_000) await fs.remove(lockPath); // stale > 1min
}
const isRepo = await isGitRepository(repoPath);

// Guard a git command invocation: inputs are a non-empty args array against a real git repo
function canRunGit(commandArgs: string[], sourceDir: string): boolean {
  return Array.isArray(commandArgs) && commandArgs.length > 0 &&
    commandArgs.every((a) => typeof a === 'string') &&
    existsSync(path.join(sourceDir, '.git'));
}

try {
  await executeGitCommandWithRetry(['git', 'commit', '-m', 'cp'], dir, 'checkpoint');
} catch (e) {
  if (e instanceof PentestError && e.code === ErrorCode.GIT_CHECKPOINT_FAILED) {
    // transient lock exhaustion — clear stale lock and retry once, else surface to Temporal
    await fs.remove(path.join(dir, '.git', 'index.lock')).catch(() => {});
  }
  throw e;
}

When it happens

Trigger: Calling executeGitCommandWithRetry (directly or via rollbackGitWorkspace / saveCheckpoint / findLatestCommit / restoreGitCheckpoint) while another git process or a sibling agent holds the repo's index.lock for longer than the entire backoff window (sum of 1s+2s+4s+8s = ~15s across 5 attempts). Also when concurrent parallel agents in the vuln/exploit phases checkpoint into the same private deliverables git repo and the global git lock context (withGitRepoLock) serializes but the OS-level .git/index.lock is stale or held.

Common situations: A crashed prior worker left a stale .git/index.lock in the workspace deliverables repo. Running two scans against the same repo/workspace simultaneously. Filesystem (NFS/Docker overlay) latency making git lock acquisition exceed the backoff window. The repo lives on a slow bind-mount inside the worker container.

Related errors

• GIT_CHECKPOINT_FAILED
• CONFIG_NOT_FOUND
• CONFIG_VALIDATION_FAILED
• Cannot create deliverables directory at ${deliverablesDir}
• Login instructions template not found

AI-assisted analysis of KeygraphHQ/shannon@1ae0a142f8 (2026-08-12). Data as JSON: /api/errors/11adb99891115794. Report an issue: GitHub.