affaan-m/ECC · error · Error

${label} must contain valid UTF-8 text.

Error message

${label} must contain valid UTF-8 text.

What it means

Thrown by decodeUtf8() when the fatal TextDecoder rejects the input buffer. The library uses TextDecoder('utf-8', { fatal: true }) (FATAL_UTF8_DECODER) rather than the lenient default, because memory documents are security-relevant state and silently substituting replacement characters (U+FFFD) could hide tampering or corruption. The label argument identifies what was being decoded (e.g. 'memory document').

Source

Thrown at scripts/lib/memory-vault-format.js:218

    updatedAt: validateTimestamp(memory.updatedAt, 'updated_at'),
    body: normalizeBody(memory.body),
  };
}

function serializeMemoryDocument(memory) {
  const normalized = normalizeMemory(memory);
  const metadata = FRONTMATTER_FIELDS.map(([serializedKey, objectKey]) => (
    `${serializedKey}: ${JSON.stringify(normalized[objectKey])}`
  )).join('\n');
  const body = normalized.body.length > 0 ? `\n\n${normalized.body}` : '';
  return `---\n${metadata}\n---${body}\n`;
}

function decodeUtf8(buffer, label = 'text') {
  try {
    return FATAL_UTF8_DECODER.decode(buffer);
  } catch {
    throw new Error(`${label} must contain valid UTF-8 text.`);
  }
}

function parseFrontmatterLine(line, sourcePath, seen) {
  const separator = line.indexOf(':');
  if (separator <= 0) {
    throw new Error(`Invalid memory frontmatter line in ${sourcePath}.`);
  }
  const serializedKey = line.slice(0, separator).trim();
  const objectKey = FRONTMATTER_KEYS.get(serializedKey);
  if (!objectKey) {
    throw new Error(`Unknown memory frontmatter field in ${sourcePath}.`);
  }
  if (seen.has(objectKey)) {
    throw new Error(`Duplicate memory frontmatter field in ${sourcePath}.`);
  }
  const rawValue = line.slice(separator + 1).trim();
  try {

View on GitHub (pinned to 01e15490f0)

Solutions

1. Re-save the offending file as UTF-8 with a text editor (use iconv -f latin-1 -t utf-8 file or recode UTF-8 file).
2. Restore the file from version control or backup; treat non-UTF-8 vault contents as corruption.
3. Run doctorMemoryVault() to enumerate which files are unreadable — they appear in invalidFiles.
4. If you control the producer, ensure it writes with encoding: 'utf8' and never mixes Buffer concat from non-UTF-8 sources.

Example fix

// before: file saved as cp1252, decode throws
const source = readRegularTextFile(filePath, { trustedRoot: root });

// after: re-encode on disk first
// shell: iconv -f cp1252 -t utf-8 file.md > file.md.utf8 && mv file.md.utf8 file.md
const source = readRegularTextFile(filePath, { trustedRoot: root });

const buffer = fs.readFileSync(filePath);
try {
  new TextDecoder('utf-8', { fatal: true }).decode(buffer);
} catch {
  throw new Error(`file ${filePath} is not valid UTF-8; re-save before importing`);
}

try {
  readRegularTextFile(filePath, { trustedRoot: root });
} catch (err) {
  if (/must contain valid UTF-8 text/.test(err.message)) {
    // quarantine the file and log; do not fall back to lenient decode
    throw new Error(`vault file ${filePath} is corrupt; restore from backup`);
  }
  throw err;
}

When it happens

Trigger: A memory .md file on disk has been corrupted by a partial write, a disk error, or an editor that saved in latin-1/cp1252 instead of UTF-8. A vault file was patched in place by a tool that injected raw bytes. readRegularTextFile() opens the file with O_NOFOLLOW and reads bytes, then decodeUtf8() converts — any non-shortest-form UTF-8 or invalid continuation byte triggers this.

Common situations: Power failure mid-write left a truncated multi-byte sequence. Windows editor saved as 'ANSI' instead of 'UTF-8'. Pipe through iconv with the wrong source encoding. Diff/patch tool that operated in byte mode and produced invalid sequences.

Related errors

• Standard input remained unavailable after ${maxRetryWaitMs}m
• memory body is too large (maximum ${maxBytes} bytes).
• Failed to read ${label}: ${error.message}
• Failed to parse ${label} at ${filePath}: ${error.message}
• ${label} must be a non-empty string.

AI-assisted analysis of affaan-m/ECC@01e15490f0 (2026-08-13). Data as JSON: /api/errors/51f78617e7fcae60. Report an issue: GitHub.