Struct Scanner

pub struct Scanner<'a> { /* private fields */ }

Converts text into a stream of tokens.

Implementations

impl<'a> Scanner<'a>

pub fn new(file_text: &'a str) -> Scanner<'a>

Creates a new scanner based on the provided text.

pub fn file_text(&self) -> &str

pub fn scan(&mut self) -> Result<Option<Token<'a>>, ParseError>

Moves to and returns the next token.

pub fn token_start(&self) -> usize

Gets the start position of the token.

pub fn token_end(&self) -> usize

Gets the end position of the token.

pub fn token(&self) -> Option<Token<'a>>

Gets the current token.