{"record":{"id":"51f78617e7fcae60","repo":"affaan-m/ECC","slug":"label-must-contain-valid-utf-8-text","errorCode":null,"errorMessage":"${label} must contain valid UTF-8 text.","messagePattern":"(.+?) must contain valid UTF-8 text\\.","errorType":"validation","errorClass":"Error","httpStatus":null,"severity":"error","filePath":"scripts/lib/memory-vault-format.js","lineNumber":218,"sourceCode":"    updatedAt: validateTimestamp(memory.updatedAt, 'updated_at'),\n    body: normalizeBody(memory.body),\n  };\n}\n\nfunction serializeMemoryDocument(memory) {\n  const normalized = normalizeMemory(memory);\n  const metadata = FRONTMATTER_FIELDS.map(([serializedKey, objectKey]) => (\n    `${serializedKey}: ${JSON.stringify(normalized[objectKey])}`\n  )).join('\\n');\n  const body = normalized.body.length > 0 ? `\\n\\n${normalized.body}` : '';\n  return `---\\n${metadata}\\n---${body}\\n`;\n}\n\nfunction decodeUtf8(buffer, label = 'text') {\n  try {\n    return FATAL_UTF8_DECODER.decode(buffer);\n  } catch {\n    throw new Error(`${label} must contain valid UTF-8 text.`);\n  }\n}\n\nfunction parseFrontmatterLine(line, sourcePath, seen) {\n  const separator = line.indexOf(':');\n  if (separator <= 0) {\n    throw new Error(`Invalid memory frontmatter line in ${sourcePath}.`);\n  }\n  const serializedKey = line.slice(0, separator).trim();\n  const objectKey = FRONTMATTER_KEYS.get(serializedKey);\n  if (!objectKey) {\n    throw new Error(`Unknown memory frontmatter field in ${sourcePath}.`);\n  }\n  if (seen.has(objectKey)) {\n    throw new Error(`Duplicate memory frontmatter field in ${sourcePath}.`);\n  }\n  const rawValue = line.slice(separator + 1).trim();\n  try {","sourceCodeStart":200,"sourceCodeEnd":236,"githubUrl":"https://github.com/affaan-m/ECC/blob/8321021c54d670126ce3b2969d5deb880b4b0c2a/scripts/lib/memory-vault-format.js#L200-L236","documentation":"decodeUtf8() decodes raw bytes (from a file read or stdin) into a UTF-8 string using a fatal decoder. Node's TextDecoder throws on invalid byte sequences when fatal:true, and this wrapper converts that into a labeled error so the caller knows which input (file path or 'stdin') contained the invalid bytes.","triggerScenarios":"Reading a memory file that is not valid UTF-8 (Latin-1/Windows-1252 encoded, binary content, truncated multi-byte sequence at the end of a cut-off file, or a BOM-mangled/UTF-16 encoded file) via readRegularTextFile, or piping non-UTF-8 bytes into the CLI via readBoundedStdin.","commonSituations":"A memory file created or edited on Windows with a non-UTF-8 codepage; a file downloaded as binary; output from a tool that emits UTF-16; a pipe carrying compressed or binary data instead of text.","solutions":["Re-encode the file as UTF-8 (iconv -f WINDOWS-1252 -t UTF-8 file, or re-save with 'UTF-8' encoding in the editor).","Check the encoding with `file -i <path>` and convert accordingly before loading.","If stdin is the source, ensure the producer emits UTF-8 text and is not piping binary/compressed data.","If the file is truncated, regenerate it — a cut multi-byte sequence at EOF is invalid UTF-8."],"exampleFix":"// before\n$ cat memory.win1252.md | ecc-memory save   # Windows-1252 bytes\n\n// after\n$ iconv -f WINDOWS-1252 -t UTF-8 memory.win1252.md > memory.utf8.md\n$ ecc-memory save < memory.utf8.md","handlingStrategy":"validation","validationCode":"const bytes = fs.readFileSync(path);\nnew TextDecoder('utf-8', { fatal: true }).decode(bytes); // throws locally if invalid\n// or: file -i path must report charset=utf-8","typeGuard":"const isValidUtf8 = (buf) => { try { new TextDecoder('utf-8', { fatal: true }).decode(buf); return true; } catch { return false; } };","tryCatchPattern":"try {\n  await saveFromFile(path);\n} catch (err) {\n  if (err.message.includes('must contain valid UTF-8 text')) {\n    console.error('%s is not valid UTF-8; convert it with iconv before saving.', path);\n  } else throw err;\n}","preventionTips":["Save all memory files as UTF-8 (no UTF-16, no BOM issues).","Run `file -i <path>` on files from Windows or downloads before loading.","Never pipe binary or compressed output into the memory CLI stdin."],"tags":["encoding","utf-8","file-input"],"backgroundTag":"invalid-argument-format","analyzedSha":"8321021c54d670126ce3b2969d5deb880b4b0c2a","analyzedAt":"2026-09-16T10:08:13.343Z","contentChangedAt":"2026-09-16T10:08:13.343Z","schemaVersion":2},"datasetVersion":"2026-09-23T08:17:48.524Z"}