{"record":{"id":"0a18f740b6b7ac01","repo":"influxdata/influxdb","slug":"error-converting-json-output-to-utf-8","errorCode":null,"errorMessage":"Error converting JSON output to UTF-8: {}","messagePattern":"Error converting JSON output to UTF-8: (.+?)","errorType":"exception","errorClass":"Error","httpStatus":null,"severity":"error","filePath":"core/influxdb_iox_client/src/format.rs","lineNumber":38,"sourceCode":"\n    /// Error pretty printing\n    #[error(\"Arrow pretty printing error: {}\", .0)]\n    PrettyArrow(ArrowError),\n\n    /// Error during CSV conversion\n    #[error(\"Arrow csv printing error: {}\", .0)]\n    CsvArrow(ArrowError),\n\n    /// Error during JSON conversion\n    #[error(\"Arrow json printing error: {}\", .0)]\n    JsonArrow(ArrowError),\n\n    /// Error converting CSV output to utf-8\n    #[error(\"Error converting CSV output to UTF-8: {}\", .0)]\n    CsvUtf8(std::string::FromUtf8Error),\n\n    /// Error converting JSON output to utf-8\n    #[error(\"Error converting JSON output to UTF-8: {}\", .0)]\n    JsonUtf8(std::string::FromUtf8Error),\n}\ntype Result<T, E = Error> = std::result::Result<T, E>;\n\n#[derive(Debug, Copy, Clone, PartialEq, Eq, Default)]\n/// Requested output format for the query endpoint\npub enum QueryOutputFormat {\n    /// Arrow pretty printer format (default)\n    #[default]\n    Pretty,\n    /// Comma separated values\n    Csv,\n    /// Arrow JSON format\n    Json,\n}\n\nimpl Display for QueryOutputFormat {\n    fn fmt(&self, f: &mut std::fmt::Formatter<'_>) -> std::fmt::Result {","sourceCodeStart":20,"sourceCodeEnd":56,"githubUrl":"https://github.com/influxdata/influxdb/blob/06200ef96ba82c5f6727e5038a83af8e722c6875/core/influxdb_iox_client/src/format.rs#L20-L56","documentation":"This error is returned when the JSON byte output produced by the arrow JSON writer cannot be converted into a Rust UTF-8 String (std::string::FromUtf8Error). The FromUtf8Error is wrapped and shown after 'Error converting JSON output to UTF-8: '.","triggerScenarios":"Calling format with Format::Json when serialized output bytes contain invalid UTF-8 sequences.","commonSituations":"Data containing malformed string bytes that survive arrow JSON serialization but fail String conversion; corrupted buffers from mismatched arrow versions.","solutions":["Sanitize/validate string column data for valid UTF-8 before export.","Investigate the byte offset reported by FromUtf8Error to find the bad record.","Use lossy conversion in a custom export path.","Verify arrow crate versions match across dependencies."],"exampleFix":"// before\nlet json = result.format(format::Format::Json)?;\n// after\nlet json = result.format(format::Format::Json)\n    .map_err(|format::Error::JsonUtf8(e)| anyhow!(\"bad utf8 in json output: {e}\"))?;","handlingStrategy":"validation","validationCode":null,"typeGuard":"fn is_json_utf8_err(e: &format::Error) -> Option<&std::string::FromUtf8Error> {\n    match e { format::Error::JsonUtf8(u) => Some(u), _ => None }\n}","tryCatchPattern":null,"preventionTips":["Validate string columns for UTF-8 before querying/exporting","Use from_utf8_lossy in custom serialization paths to tolerate bad bytes","Keep arrow versions consistent to avoid corrupted output buffers"],"tags":["utf-8","json","encoding"],"backgroundTag":"invalid-argument-format","analyzedSha":"06200ef96ba82c5f6727e5038a83af8e722c6875","analyzedAt":"2026-09-19T12:55:30.003Z","contentChangedAt":"2026-09-19T12:55:30.003Z","schemaVersion":2},"datasetVersion":"2026-09-23T08:17:48.524Z"}