influxdata/influxdb · error · Error

Error converting JSON output to UTF-8

Error message

Error converting JSON output to UTF-8: {}

What it means

This error is returned when the JSON byte output produced by the arrow JSON writer cannot be converted into a Rust UTF-8 String (std::string::FromUtf8Error). The FromUtf8Error is wrapped and shown after 'Error converting JSON output to UTF-8: '.

Solutions

1. Sanitize/validate string column data for valid UTF-8 before export.
2. Investigate the byte offset reported by FromUtf8Error to find the bad record.
3. Use lossy conversion in a custom export path.
4. Verify arrow crate versions match across dependencies.

Example fix

// before
let json = result.format(format::Format::Json)?;
// after
let json = result.format(format::Format::Json)
    .map_err(|format::Error::JsonUtf8(e)| anyhow!("bad utf8 in json output: {e}"))?;

fn is_json_utf8_err(e: &format::Error) -> Option<&std::string::FromUtf8Error> {
    match e { format::Error::JsonUtf8(u) => Some(u), _ => None }
}

When it happens

Trigger: Calling format with Format::Json when serialized output bytes contain invalid UTF-8 sequences.

Common situations: Data containing malformed string bytes that survive arrow JSON serialization but fail String conversion; corrupted buffers from mismatched arrow versions.

Understand the failure class

Background: "Invalid ... format", "must be in format X", "does not look like a ..." — invalid argument format errors across CLI tools and libraries — this error's family across 17 libraries.

Related errors

• Error converting CSV output to UTF-8
• Arrow json printing error
• body content is not valid utf8
• deserializing show tables
• error decoding query body

AI-assisted analysis of influxdata/influxdb@06200ef96b (2026-09-19). Data as JSON: /api/errors/0a18f740b6b7ac01. Report an issue: GitHub.

Appendix: source

Thrown at core/influxdb_iox_client/src/format.rs:38

    /// Error pretty printing
    #[error("Arrow pretty printing error: {}", .0)]
    PrettyArrow(ArrowError),

    /// Error during CSV conversion
    #[error("Arrow csv printing error: {}", .0)]
    CsvArrow(ArrowError),

    /// Error during JSON conversion
    #[error("Arrow json printing error: {}", .0)]
    JsonArrow(ArrowError),

    /// Error converting CSV output to utf-8
    #[error("Error converting CSV output to UTF-8: {}", .0)]
    CsvUtf8(std::string::FromUtf8Error),

    /// Error converting JSON output to utf-8
    #[error("Error converting JSON output to UTF-8: {}", .0)]
    JsonUtf8(std::string::FromUtf8Error),
}
type Result<T, E = Error> = std::result::Result<T, E>;

#[derive(Debug, Copy, Clone, PartialEq, Eq, Default)]
/// Requested output format for the query endpoint
pub enum QueryOutputFormat {
    /// Arrow pretty printer format (default)
    #[default]
    Pretty,
    /// Comma separated values
    Csv,
    /// Arrow JSON format
    Json,
}

impl Display for QueryOutputFormat {
    fn fmt(&self, f: &mut std::fmt::Formatter<'_>) -> std::fmt::Result {

View on GitHub (pinned to 06200ef96b)