tursodatabase/turso · error · std::io::Error

InvalidData

InvalidData

Error message

non-UTF8 SQL: {e}

What it means

The tursodb CLI buffers stdin until a ';' and must convert the statement bytes to a Rust string; if they are not valid UTF-8 it fails fast with io::ErrorKind::InvalidData instead of executing mojibake SQL.

Source

Thrown at cli/input.rs:288

        // Skip if buffer empty or no ';'
        if let Some(semicolon_pos) = self.buf.iter().rposition(|&b| b == b';') {
            // Are all bytes after ';' ASCII whitespace?
            if self.buf[semicolon_pos + 1..]
                .iter()
                .all(|&b| matches!(b, b' ' | b'\t' | b'\r' | b'\n'))
            {
                let stmt_bytes = self.buf[..=semicolon_pos].to_vec();
                self.buf.clear();
                self.exec_stmt_bytes(&stmt_bytes)?;
            }
        }
        Ok(())
    }

    fn exec_stmt_bytes(&self, stmt_bytes: &[u8]) -> io::Result<()> {
        // SQL must be UTF-8. If not, surface a clear error.
        let sql = std::str::from_utf8(stmt_bytes).map_err(|e| {
            io::Error::new(io::ErrorKind::InvalidData, format!("non-UTF8 SQL: {e}"))
        })?;
        self.exec_stmt(sql)
            .map_err(|e| io::Error::other(e.to_string()))
    }

    fn exec_stmt(&self, sql: &str) -> Result<(), LimboError> {
        match self.target.query(sql) {
            Ok(Some(mut rows)) => {
                rows.run_with_row_callback(|_| Ok(()))?;
            }
            Ok(None) => {}
            Err(e) => return Err(e),
        }
        Ok(())
    }
}

impl<'a> Write for ApplyWriter<'a> {

View on GitHub (pinned to 492c4a71cd)

Solutions

1. Convert the input to UTF-8: `iconv -f UTF-16 -t UTF-8 dump.sql | tursodb`
2. Decompress first: `gunzip -c dump.sql.gz | tursodb`
3. Strip a BOM with `sed '1s/^\xEF\xBB\xBF//'` or an editor
4. Verify with `file dump.sql` before piping

Example fix

# before (PowerShell emits UTF-16)
Get-Content dump.sql | tursodb
# after
iconv -f UTF-16 -t UTF-8 dump.sql | tursodb

# verify encoding before piping
file dump.sql
iconv -f UTF-8 -t UTF-8 dump.sql >/dev/null && echo utf8-ok

When it happens

Trigger: Piping SQL that is not UTF-8: latin-1/CP1252 dumps, UTF-16 files (e.g. PowerShell redirects), BOM-prefixed UTF-16, or binary/gzip files fed without decompression.

Common situations: `cat dump.sql | tursodb` where dump.sql came from Windows tooling, exports in legacy encodings, or accidentally piping a .gz archive.

Related errors

• Error: file "{}" is not valid UTF-8 text – {}
• Failed to read manual page: {}
• Unexpected row for INSERT
• setAsciiStream length must be non-negative
• setUnicodeStream length must be non-negative

AI-assisted analysis of tursodatabase/turso@492c4a71cd (2026-08-20). Data as JSON: /api/errors/45649493e0de678c. Report an issue: GitHub.