paperclipai/paperclip · error · Error

ACPX ${agent} runtime executable changed while it was verifi

Error message

ACPX ${agent} runtime executable changed while it was verified

What it means

While hashing, the runtime re-stat's the open handle and re-lstat's the path, comparing device/inode identities across three snapshots (lexical before, lexical after, handle after). If any identity differs — or the file disappeared, became a symlink, or could not be fully read (position !== before.size) — the file mutated during verification, so the digest cannot be trusted and this error is thrown. This is a deliberate race-detection guard against binary-swapping attacks.

Source

Thrown at packages/paperclip-runner/src/drivers/acpx/installation-integrity.ts:1088

    } finally {
      buffer.fill(0);
    }
    const after = await handle.stat({ bigint: true });
    const lexicalAfter = await lstat(executablePath, { bigint: true }).catch(
      () => null,
    );
    const beforeIdentity = fileIdentity(before);
    const afterIdentity = fileIdentity(after);
    if (
      position !== Number(before.size) ||
      lexicalAfter === null ||
      lexicalAfter.isSymbolicLink() ||
      !lexicalAfter.isFile() ||
      !sameIdentity(fileIdentity(lexicalBefore), fileIdentity(lexicalAfter)) ||
      !sameIdentity(fileIdentity(lexicalAfter), afterIdentity) ||
      !sameIdentity(beforeIdentity, afterIdentity)
    ) {
      throw new Error(
        `ACPX ${agent} runtime executable changed while it was verified`,
      );
    }
    const digest = `sha256:${hash.digest("hex")}`;
    if (digest !== expectedDigest) {
      throw new Error(`ACPX ${agent} runtime executable digest mismatch`);
    }
    return { handle, identity: afterIdentity };
  } catch (error) {
    await handle.close();
    throw error;
  }
}

/** Fail closed where Node cannot atomically refuse a final symlink component. */
export function verifiedExecutableOpenFlags(
  platform: NodeJS.Platform,
  noFollowFlag: number | undefined,

View on GitHub (pinned to 01ad858492)

Solutions

  1. Retry the runtime start once the concurrent install/upgrade process has finished — this is usually a transient race
  2. Serialize runtime upgrades with runtime starts (a lock around install + spawn) so the binary is never swapped mid-verification
  3. Pin the runtime version and disable auto-updaters/AV quarantine on the install directory
  4. Install the binary atomically (write to temp + `rename`) so readers never observe a half-swapped file
  5. Check for competing processes (CI jobs, second service instance) touching the same install path

Example fix

// before: naive in-place upgrade racing with spawn
curl -o /opt/acpx/bin/acpx https://...   # spawn may read mid-write
// after: atomic install + lock
acquireLock('acpx-install')
const tmp = '/opt/acpx/bin/.acpx.tmp'; writeDownload(tmp); chmod +x tmp
renameSync(tmp, '/opt/acpx/bin/acpx')
releaseLock(); thenStartRuntime()
Defensive patterns

Strategy: retry

Try / catch

try {
  await startAcppRuntime();
} catch (e) {
  if (String(e?.message).includes('changed while it was verified')) {
    // file was swapped mid-verification; wait for quiescence and retry once
    await waitForInstallLockRelease();
    await startAcppRuntime();
  } else {
    throw e;
  }
}

Prevention

When it happens

Trigger: The executable file is replaced, deleted, resized, or hard-link-swapped by another process between the initial lstat and the post-hash checks; the read loop ends early leaving `position !== Number(before.size)`; or lstat after hashing returns null (file removed).

Common situations: Package manager or auto-updater upgrading the binary at the exact moment a run starts; two installers racing on the same install dir; cleanup/AV quarantine replacing the file; container image being rebuilt while the service runs; CI jobs re-linking binaries concurrently.

Understand the failure class

Background: "This is a bug, please report it": internal invariant violations, unreachable panics, and SNH errors explained — this error's family across 47 libraries.

Related errors


AI-assisted analysis of paperclipai/paperclip@01ad858492 (2026-09-10). Data as JSON: /api/errors/2cf11a1924ee914d. Report an issue: GitHub.