databendlabs/databend · error · InvalidData

Expected to have quotes in string literal.

Error message

Expected to have quotes in string literal.

What it means

Raised by the `check_pos` helper used by `fast_read_quoted_text`. The fast path pre-records the byte positions of quote/escape characters and verifies them while rescanning; if the cursor's current position does not match any recorded quote position, the literal is assumed malformed and this InvalidData error is returned.

Solutions

  1. Fall back to the non-fast `read_quoted_text` (via `read_string_text`) which is more tolerant and gives better diagnostics.
  2. Verify the literal is valid UTF-8 and uses a single consistent quote character.
  3. Check for unbalanced or unescaped quotes/escapes in the offending literal.
  4. If parsing third-party text, pre-normalize it (re-escape quotes) before calling the fast parser.

Example fix

// before: fast path on suspicious input
let s = "'weird\\\"literal".as_bytes().fast_read_quoted_text(&mut buf)?;
// after: use the robust reader for untrusted input
let s = "'weird\\\"literal'".as_bytes().read_quoted_text(&mut buf)?;
Defensive patterns

Strategy: fallback

Validate before calling

fn valid_utf8_literal(input: &[u8]) -> bool {
    std::str::from_utf8(input).is_ok()
}

Type guard

fn usable_by_fast_path(input: &[u8]) -> bool {
    std::str::from_utf8(input).map(|s| {
        s.starts_with('\'') && s.ends_with('\'')
    }).unwrap_or(false)
}

Try / catch

let result = match fast_read_quoted_text(&mut buf) {
    Ok(s) => Ok(s),
    Err(_) => input.as_bytes().read_quoted_text(&mut buf), // tolerant fallback
};

Prevention

When it happens

Trigger: Calling `fast_read_quoted_text` on input whose escape/quote structure is inconsistent — e.g. the scanner lands on a position that was not registered as a quote, which happens with malformed escapes, unbalanced quotes, or bytes that altered the scan trajectory (multi-byte/invalid UTF-8 sequences in the middle of the literal).

Common situations: String literals containing invalid UTF-8, mixed quote styles, or hand-edited SQL where an escape was removed but its target position stayed — any case where the fast scanner's recorded quote positions no longer line up with the actual input.

Related errors

AI-assisted analysis of databendlabs/databend@288d84d76e (2026-09-11). Data as JSON: /api/errors/88bd40ecece7121a. Report an issue: GitHub.

Appendix: source

Thrown at src/common/io/src/cursor_ext/cursor_read_string_ext.rs:198

        }
        Err(std::io::Error::new(
            ErrorKind::InvalidData,
            format!(
                "Expected to have terminated string literal after quota \', while consumed buf: {:?}",
                buf
            ),
        ))
    }
}

// Check that the pre-calculated position is correct.
fn check_pos(curr_pos: usize, positions: &mut VecDeque<usize>) -> Result<()> {
    if let Some(pos) = positions.pop_front()
        && curr_pos == pos
    {
        return Ok(());
    }
    Err(std::io::Error::new(
        ErrorKind::InvalidData,
        "Expected to have quotes in string literal.".to_string(),
    ))
}

fn unescape(c: u8) -> u8 {
    match c {
        b'a' => b'\x07', // \a in c
        b'b' => b'\x08', // \b in c
        b'v' => b'\x0B', // \v in c
        b'f' => b'\x0C', // \f in c
        b'e' => b'\x1B', // \e in c
        b'n' => b'\n',
        b'r' => b'\r',
        b't' => b'\t',
        b'0' => b'\0',
        _ => c,
    }

View on GitHub (pinned to 288d84d76e)