santifer/career-ops · warning

gmail: could not persist processed-id state

Error message

gmail: could not persist processed-id state — ${err.message}

What it means

Warning in plugins/gmail/index.mjs's saveProcessedIds: persisting the processed Gmail message IDs to STATE_PATH (JSON in data/) failed. The IDs are then re-processed on the next ingest run, causing duplicate work; the error is logged and swallowed so the current ingest continues.

Solutions

  1. Check err.message for the fs error code (EACCES/ENOSPC/EPERM)
  2. Ensure the working directory is writable: mkdir -p data and verify permissions
  3. Avoid concurrent plugin runs sharing the same state file, or add file locking
  4. Re-run ingest; duplicates from the failed save are re-fetched and re-classified next time

Example fix

// before
mkdirSync('data', { recursive: true });
writeFileSync(STATE_PATH, JSON.stringify({ processed_message_ids: [...ids] }, null, 2), 'utf-8');
// after
mkdirSync(dirname(STATE_PATH), { recursive: true });
const tmp = STATE_PATH + '.tmp';
writeFileSync(tmp, JSON.stringify({ processed_message_ids: [...ids] }, null, 2), 'utf-8');
renameSync(tmp, STATE_PATH); // atomic, avoids partial state on crash
Defensive patterns

Strategy: try-catch

Validate before calling

import { accessSync, constants } from 'fs';
try { accessSync('data', constants.W_OK); } catch { mkdirSync('data', { recursive: true }); }

Type guard

function isFsError(e) { return e instanceof Error && typeof e.code === 'string'; }

Try / catch

try { atomicWriteJson(STATE_PATH, { processed_message_ids: [...ids] }); } catch (err) { console.warn(`gmail: could not persist processed-id state — ${err.message}`); }

Prevention

When it happens

Trigger: writeFileSync to data/.gmail-state.json fails: data/ cannot be created (EACCES on mkdirSync despite recursive), disk full, read-only filesystem, or file locked by another process.

Common situations: Running the plugin in a sandbox/CI where the repo is read-only; concurrent plugin runs writing the state file simultaneously; disk quota exceeded.

Understand the failure class

Background: "failed to write file", "Could not save figure", "Error saving remote file" — file write failed: causes and fixes across languages and libraries — this error's family across 38 libraries.

Related errors

AI-assisted analysis of santifer/career-ops@aac998c7ed (2026-09-16). Data as JSON: /api/errors/00d469ee16d91e89. Report an issue: GitHub.

Appendix: source

Thrown at plugins/gmail/index.mjs:66

  return data.access_token;
}

function loadProcessedIds() {
  if (!existsSync(STATE_PATH)) return new Set();
  try {
    const state = JSON.parse(readFileSync(STATE_PATH, 'utf-8'));
    return new Set(state.processed_message_ids || []);
  } catch {
    return new Set();
  }
}

function saveProcessedIds(ids) {
  try {
    mkdirSync('data', { recursive: true });
    writeFileSync(STATE_PATH, JSON.stringify({ processed_message_ids: [...ids] }, null, 2), 'utf-8');
  } catch (err) {
    console.warn(`gmail: could not persist processed-id state — ${err.message}`);
  }
}

/** @type {{ ingest: (ctx: any) => Promise<object[]> }} */
export default {
  async ingest(ctx) {
    const clientId = ctx?.env?.GMAIL_CLIENT_ID;
    const clientSecret = ctx?.env?.GMAIL_CLIENT_SECRET;
    const refreshToken = ctx?.env?.GMAIL_REFRESH_TOKEN;
    if (!clientId || !clientSecret || !refreshToken) {
      throw new Error('gmail: missing GMAIL_CLIENT_ID / GMAIL_CLIENT_SECRET / GMAIL_REFRESH_TOKEN in .env');
    }

    const label = ctx?.settings?.label || 'Job Leads';
    const daysBack = Number(ctx?.settings?.days_back ?? 7);
    if (!Number.isInteger(daysBack) || daysBack <= 0) {
      throw new Error(`gmail: invalid days_back "${ctx?.settings?.days_back}" (must be a positive integer)`);
    }

View on GitHub (pinned to aac998c7ed)