[verify-skills-cli-runtime] a dry-run reported execution

Error message

[verify-skills-cli-runtime] a dry-run reported execution

What it means

Thrown by verifySkillsCliRuntime() when a skills install --dry-run or skills update --dry-run command reports executed !== false. The script asserts that dry-run mode does not perform any filesystem mutation — both install.executed and update.executed must be exactly false. This catches dry-run leaks where a code path performs real writes despite the --dry-run flag.

Source

Thrown at config/scripts/verify-skills-cli-runtime.cjs:210

  const install = parseJson(
    'skills install --dry-run',
    runCli(absoluteOutDir, [
      'skills',
      'install',
      '--skill',
      'orca-cli',
      '--agent',
      'codex',
      '--dry-run',
      '--json'
    ])
  )
  const update = parseJson(
    'skills update --dry-run',
    runCli(absoluteOutDir, ['skills', 'update', '--skill', 'orca-cli', '--dry-run', '--json'])
  )
  if (install.executed !== false || update.executed !== false) {
    throw new Error('[verify-skills-cli-runtime] a dry-run reported execution')
  }

  return { closureFiles: closure.length, commands: 5 }
}

if (require.main === module) {
  try {
    const result = verifySkillsCliRuntime(process.argv[2] ?? 'out')
    console.log(
      `[verify-skills-cli-runtime] ${result.closureFiles} closure files and ` +
        `${result.commands} commands passed`
    )
  } catch (error) {
    console.error(error instanceof Error ? error.message : error)
    process.exitCode = 1
  }
}

View on GitHub (pinned to 1136503c6a)

Solutions

Run skills install --skill orca-cli --agent codex --dry-run --json and inspect the executed field — confirm whether it is true, undefined, or under a different key.
Trace the --dry-run flag through the install/update code path to the point where files would be written; ensure the guard short-circuits before any mutation.
If the JSON schema changed (e.g. 'executed' → 'didExecute'), update the assertion in the verifier.
Ensure executed is explicitly set to false in dry-run mode, not left undefined.

Example fix

// before: dry-run flag checked after write
await fs.writeFile(path, content)
if (options.dryRun) return { executed: false }

// after: guard before any mutation
if (options.dryRun) return { executed: false }
await fs.writeFile(path, content)

Defensive patterns

Strategy: validation

Validate before calling

function assertDryRun(result, label) {
  if (result.executed !== false) {
    throw new Error(
      `${label} dry-run reported executed=${String(result.executed)} (expected false)`
    )
  }
}

Type guard

function isDryRunResult(value) {
  return value !== null && typeof value === 'object' &&
    'executed' in value && typeof value.executed === 'boolean'
}

Try / catch

try {
  const install = parseJson('skills install --dry-run', installOutput)
  const update = parseJson('skills update --dry-run', updateOutput)
  assertDryRun(install, 'install')
  assertDryRun(update, 'update')
} catch (err) {
  // A dry-run writing files is a correctness bug — fail the build
  throw err
}

Prevention

Gate all filesystem writes behind an `if (!options.dryRun)` check at the earliest point.
Add a unit test that runs dry-run against a temp dir and asserts no files were created.
Audit new code paths added to install/update for missing dry-run guards.

When it happens

Trigger: parseJson of the --dry-run --json output yields an object where install.executed !== false or update.executed !== false. Caused by: the dry-run flag not being propagated to the write logic; executed defaulting to true or undefined; the install/update command performing the operation before checking the flag.

Common situations: A refactor that moves the dry-run check after the write; a new code path added without a dry-run guard; executed being set based on a condition that is true in the test environment (e.g. a file already exists); the JSON schema changing so executed is reported under a different key.

Related errors

AI-assisted analysis of stablyai/orca@1136503c6a (2026-08-12). Data as JSON: /api/errors/d13dcc20d1b4b45f. Report an issue: GitHub.