{"record":{"id":"b3606514a6586f2c","repo":"astrid-runtime/astrid","slug":"invalid-utf-8-sequence-of-n-bytes-from-index-m","errorCode":null,"errorMessage":"invalid utf-8 sequence of {N} bytes from index {M} (wrapped Utf8Error from pair-token store file)","messagePattern":"invalid utf-8 sequence of (.+?) bytes from index (.+?) \\(wrapped Utf8Error from pair-token store file\\)","errorType":"validation","errorClass":null,"httpStatus":null,"severity":"error","filePath":"crates/astrid-kernel/src/pair_token/mod.rs","lineNumber":504,"sourceCode":"        {\n            let _ = &self.path;\n            return Ok(Vec::new());\n        }\n        #[cfg(not(all(target_arch = \"wasm32\", target_os = \"unknown\")))]\n        {\n            self.load_from_disk()\n        }\n    }\n\n    #[cfg(not(all(target_arch = \"wasm32\", target_os = \"unknown\")))]\n    fn load_from_disk(&self) -> Result<Vec<PairToken>, PairTokenStoreError> {\n        let bytes = match std::fs::read(&self.path) {\n            Ok(b) => b,\n            Err(e) if e.kind() == std::io::ErrorKind::NotFound => return Ok(Vec::new()),\n            Err(e) => return Err(PairTokenStoreError::Io(e)),\n        };\n        let text = std::str::from_utf8(&bytes).map_err(|e| {\n            PairTokenStoreError::Io(std::io::Error::new(std::io::ErrorKind::InvalidData, e))\n        })?;\n        if text.trim().is_empty() {\n            if let Err(error) = self.save_to_disk(&[]) {\n                warn!(\n                    path = %self.path.display(),\n                    %error,\n                    \"could not normalize empty pair-token store\"\n                );\n            }\n            return Ok(Vec::new());\n        }\n        let probe: SchemaProbe = toml::from_str(text).map_err(PairTokenStoreError::Toml)?;\n        if probe.schema_version > STORE_SCHEMA_VERSION {\n            return Err(PairTokenStoreError::Io(std::io::Error::new(\n                std::io::ErrorKind::InvalidData,\n                format!(\n                    \"pair-token store schema {} is newer than supported schema {STORE_SCHEMA_VERSION}\",\n                    probe.schema_version","sourceCodeStart":486,"sourceCodeEnd":522,"githubUrl":"https://github.com/astrid-runtime/astrid/blob/affd8760f44190dbdfbec23403f4c4b642c33112/crates/astrid-kernel/src/pair_token/mod.rs#L486-L522","documentation":"The pair-token store file is read from disk and must be valid UTF-8; std::str::from_utf8 failure is wrapped in PairTokenStoreError::Io as InvalidData, surfacing Rust's native Utf8Error message ('invalid utf-8 sequence of N bytes from index M'). NotFound is tolerated (empty store) but genuinely corrupt bytes abort the load.","triggerScenarios":"Calling PairTokenStore::load (→ load_from_disk) when the TOML file on disk contains invalid UTF-8 bytes.","commonSituations":"Partial/truncated writes from a crash; the file was edited with a binary tool or got encoding mangled; disk corruption; another process wrote bytes concurrently.","solutions":["Restore the pair-token store file from a backup or delete it to start with an empty store (tokens must be re-paired)","Re-save the file as UTF-8 if it was hand-edited with a wrong encoding","Check for concurrent writers and ensure writes are atomic (write-temp-then-rename)","Run fsck / check disk health if corruption recurs"],"exampleFix":"// before: hand-edited with latin-1 escapes\niconv -f ISO-8859-1 -t UTF-8 pair-tokens.toml -o pair-tokens.toml\n// after: or reset\nmv pair-tokens.toml pair-tokens.toml.bak  # store recreates empty file","handlingStrategy":"try-catch","validationCode":"let bytes = std::fs::read(&store_path)?;\nif std::str::from_utf8(&bytes).is_err() { eprintln!(\"pair-token store is not valid UTF-8\"); }","typeGuard":null,"tryCatchPattern":"match store.load() {\n    Err(PairTokenStoreError::Io(e)) if e.kind() == std::io::ErrorKind::InvalidData && e.to_string().contains(\"utf-8\") => {\n        // restore backup or reset store and re-pair\n    }\n    other => other?,\n}","preventionTips":["Write the store atomically (temp file + rename)","Edit only with UTF-8-capable editors","Check disk health if corruption recurs","Back up the store before upgrades"],"tags":["utf-8","encoding","file-read"],"backgroundTag":"invalid-utf8","analyzedSha":"affd8760f44190dbdfbec23403f4c4b642c33112","analyzedAt":"2026-09-09T21:28:12.402Z","contentChangedAt":"2026-09-09T21:28:12.402Z","schemaVersion":2},"datasetVersion":"2026-09-17T15:17:12.973Z"}