Struct Lexer

pub struct Lexer<'src> { /* private fields */ }

The lexer.

Breaks source text up into a sequence of tokens (with source positions).

Implementations

impl<'src> Lexer<'src>

pub fn new(file: usize, src: &'src str) -> Result<Lexer<'src>, Error>

Create a new lexer for the given source contents

pub fn pos(&self) -> Pos

Get the lexer’s current source position.

pub fn next(&mut self) -> Result<Option<(Pos, Token)>, Error>

Get the next token from this lexer’s token stream, if any.

pub fn peek(&self) -> Option<&(Pos, Token)>

Peek ahead at the next token.

pub fn eof(&self) -> bool

Are we at the end of the source input?

Trait Implementations

impl<'src> Clone for Lexer<'src>

fn clone(&self) -> Lexer<'src>

Returns a copy of the value.

fn clone_from(&mut self, source: &Self)

Performs copy-assignment from source.

impl<'src> Debug for Lexer<'src>

fn fmt(&self, f: &mut Formatter<'_>) -> Result

Formats the value using the given formatter.