{"record":{"id":"05dcfbb853b14df8","repo":"uutils/coreutils","slug":"invalidutf8","errorCode":null,"errorMessage":"InvalidUtf8","messagePattern":"InvalidUtf8","errorType":"validation","errorClass":"clap::Error","httpStatus":null,"severity":"error","filePath":"src/uucore/src/lib/features/parser/shortcut_value_parser.rs","lineNumber":100,"sourceCode":"        ContextValue::StyledStrs(vec![format!(\n            \"It looks like '{value}' could match several values. Did you mean {formatted_possible_values}?\"\n        )\n        .into()]),\n    );\n}\n\nimpl TypedValueParser for ShortcutValueParser {\n    type Value = String;\n\n    fn parse_ref(\n        &self,\n        cmd: &clap::Command,\n        arg: Option<&clap::Arg>,\n        value: &std::ffi::OsStr,\n    ) -> Result<Self::Value, clap::Error> {\n        let value = value\n            .to_str()\n            .ok_or(clap::Error::new(ErrorKind::InvalidUtf8))?;\n\n        let matched_values: Vec<_> = self\n            .0\n            .iter()\n            .filter(|x| x.get_name_and_aliases().any(|name| name.starts_with(value)))\n            .collect();\n\n        match matched_values.len() {\n            0 => Err(self.generate_clap_error(cmd, arg, value, &[])),\n            1 => Ok(matched_values[0].get_name().to_string()),\n            _ => {\n                if let Some(direct_match) = matched_values.iter().find(|x| x.get_name() == value) {\n                    Ok(direct_match.get_name().to_string())\n                } else {\n                    Err(self.generate_clap_error(cmd, arg, value, &matched_values))\n                }\n            }\n        }","sourceCodeStart":82,"sourceCodeEnd":118,"githubUrl":"https://github.com/uutils/coreutils/blob/92995feca29e324a7c559ede91db81c32d9f5d59/src/uucore/src/lib/features/parser/shortcut_value_parser.rs#L82-L118","documentation":"ShortcutValueParser::parse_ref requires the raw OsStr value to be valid UTF-8 before doing prefix matching against possible values; otherwise it returns clap::ErrorKind::InvalidUtf8. Clap's typed value parsers reject non-UTF-8 arguments by default.","triggerScenarios":"Passing an argument containing invalid UTF-8 bytes (e.g. from a shell variable with raw bytes) to a CLI option backed by this parser; also reproduced in the unit tests with non-UTF-8 input.","commonSituations":"Filenames with Latin-1 or arbitrary bytes passed as option values; scripts importing data from systems with different encodings.","solutions":["Ensure the argument is valid UTF-8 before invoking (e.g. iconv or validate in the script)","If you control the parser, use a lossy conversion or an OsStr-based matcher instead of to_str()","Catch the clap error and surface a clear message to the user"],"exampleFix":"// before\nlet value = raw_os_str;\n// after\nlet value = raw_os_str.to_str().ok_or_else(|| anyhow!(\"value must be UTF-8\"))?;","handlingStrategy":"type-guard","validationCode":"fn is_utf8(v: &std::ffi::OsStr) -> bool { v.to_str().is_some() }","typeGuard":"fn as_utf8(v: &std::ffi::OsStr) -> Option<&str> { v.to_str() }","tryCatchPattern":"match parse_ref(cmd, arg, os_value) {\n    Err(e) if e.kind() == clap::error::ErrorKind::InvalidUtf8 => eprintln!(\"argument must be valid UTF-8\"),\n    r => r?,\n}","preventionTips":["Sanitize environment variables containing raw bytes before passing them as CLI args","Convert incoming data to UTF-8 (with lossy replacement) at ingestion points","Avoid `ls | xargs` pipelines over non-UTF-8 filenames"],"tags":["cli","clap","utf8","parsing"],"backgroundTag":"invalid-argument-value","analyzedSha":"92995feca29e324a7c559ede91db81c32d9f5d59","analyzedAt":"2026-09-20T12:39:35.476Z","contentChangedAt":"2026-09-20T12:39:35.476Z","schemaVersion":2},"datasetVersion":"2026-09-23T08:17:48.524Z"}