affaan-m/ECC · error · Error

must contain valid UTF-8 text.

Error message

${label} must contain valid UTF-8 text.

What it means

decodeUtf8() decodes raw bytes (from a file read or stdin) into a UTF-8 string using a fatal decoder. Node's TextDecoder throws on invalid byte sequences when fatal:true, and this wrapper converts that into a labeled error so the caller knows which input (file path or 'stdin') contained the invalid bytes.

Solutions

1. Re-encode the file as UTF-8 (iconv -f WINDOWS-1252 -t UTF-8 file, or re-save with 'UTF-8' encoding in the editor).
2. Check the encoding with `file -i <path>` and convert accordingly before loading.
3. If stdin is the source, ensure the producer emits UTF-8 text and is not piping binary/compressed data.
4. If the file is truncated, regenerate it — a cut multi-byte sequence at EOF is invalid UTF-8.

Example fix

// before
$ cat memory.win1252.md | ecc-memory save   # Windows-1252 bytes

// after
$ iconv -f WINDOWS-1252 -t UTF-8 memory.win1252.md > memory.utf8.md
$ ecc-memory save < memory.utf8.md

const bytes = fs.readFileSync(path);
new TextDecoder('utf-8', { fatal: true }).decode(bytes); // throws locally if invalid
// or: file -i path must report charset=utf-8

const isValidUtf8 = (buf) => { try { new TextDecoder('utf-8', { fatal: true }).decode(buf); return true; } catch { return false; } };

try {
  await saveFromFile(path);
} catch (err) {
  if (err.message.includes('must contain valid UTF-8 text')) {
    console.error('%s is not valid UTF-8; convert it with iconv before saving.', path);
  } else throw err;
}

When it happens

Trigger: Reading a memory file that is not valid UTF-8 (Latin-1/Windows-1252 encoded, binary content, truncated multi-byte sequence at the end of a cut-off file, or a BOM-mangled/UTF-16 encoded file) via readRegularTextFile, or piping non-UTF-8 bytes into the CLI via readBoundedStdin.

Common situations: A memory file created or edited on Windows with a non-UTF-8 codepage; a file downloaded as binary; output from a tool that emits UTF-16; a pipe carrying compressed or binary data instead of text.

Understand the failure class

Background: "Invalid ... format", "must be in format X", "does not look like a ..." — invalid argument format errors across CLI tools and libraries — this error's family across 17 libraries.

Related errors

• approved text must be valid UTF-8 text
• ffmpeg encode failed
• ffmpeg failed
• ffmpeg grade failed
• invalid JSON artifact

AI-assisted analysis of affaan-m/ECC@8321021c54 (2026-09-16). Data as JSON: /api/errors/51f78617e7fcae60. Report an issue: GitHub.

Appendix: source

Thrown at scripts/lib/memory-vault-format.js:218

    updatedAt: validateTimestamp(memory.updatedAt, 'updated_at'),
    body: normalizeBody(memory.body),
  };
}

function serializeMemoryDocument(memory) {
  const normalized = normalizeMemory(memory);
  const metadata = FRONTMATTER_FIELDS.map(([serializedKey, objectKey]) => (
    `${serializedKey}: ${JSON.stringify(normalized[objectKey])}`
  )).join('\n');
  const body = normalized.body.length > 0 ? `\n\n${normalized.body}` : '';
  return `---\n${metadata}\n---${body}\n`;
}

function decodeUtf8(buffer, label = 'text') {
  try {
    return FATAL_UTF8_DECODER.decode(buffer);
  } catch {
    throw new Error(`${label} must contain valid UTF-8 text.`);
  }
}

function parseFrontmatterLine(line, sourcePath, seen) {
  const separator = line.indexOf(':');
  if (separator <= 0) {
    throw new Error(`Invalid memory frontmatter line in ${sourcePath}.`);
  }
  const serializedKey = line.slice(0, separator).trim();
  const objectKey = FRONTMATTER_KEYS.get(serializedKey);
  if (!objectKey) {
    throw new Error(`Unknown memory frontmatter field in ${sourcePath}.`);
  }
  if (seen.has(objectKey)) {
    throw new Error(`Duplicate memory frontmatter field in ${sourcePath}.`);
  }
  const rawValue = line.slice(separator + 1).trim();
  try {

View on GitHub (pinned to 8321021c54)