Trait sqlite3_parser::lexer::Splitter

pub trait Splitter: Sized {
    type Error: ScanError;
    type TokenType;

    // Required method
    fn split<'input>(
        &mut self,
        data: &'input [u8],
    ) -> Result<(Option<(&'input [u8], Self::TokenType)>, usize), Self::Error>;
}

Split function used to tokenize the input

Required Associated Types

type Error: ScanError

Potential error raised

type TokenType

Token generated

Required Methods

fn split<'input>( &mut self, data: &'input [u8], ) -> Result<(Option<(&'input [u8], Self::TokenType)>, usize), Self::Error>

The arguments are an initial substring of the remaining unprocessed data.

If the returned error is non-nil, scanning stops and the error is returned to the client.

The function is never called with an empty data slice.

Object Safety

This trait is not object safe.

Implementors

impl Splitter for Tokenizer

use sqlite3_parser::lexer::sql::Tokenizer;
use sqlite3_parser::lexer::Scanner;

let tokenizer = Tokenizer::new();
let input = b"PRAGMA parser_trace=ON;";
let mut s = Scanner::new(tokenizer);
let Ok((_, Some((token1, _)), _)) = s.scan(input) else { panic!() };
s.scan(input).unwrap();
assert!(b"PRAGMA".eq_ignore_ascii_case(token1));

type Error = Error

type TokenType = TokenType