Crate logos

Logos

Create ridiculously fast Lexers.

Logos has two goals:

• To make it easy to create a Lexer, so you can focus on more complex problems.
• To make the generated Lexer faster than anything you’d write by hand.

To achieve those, Logos:

• Combines all token definitions into a single deterministic state machine.
• Optimizes branches into lookup tables or jump tables.
• Prevents backtracking inside token definitions.
• Unwinds loops, and batches reads to minimize bounds checking.
• Does all of that heavy lifting at compile time.

See the Logos handbook for additional documentation and usage examples.

Re-exports

pub use crate::source::Source;

Modules

source

This module contains a bunch of traits necessary for processing byte strings.

Structs

Lexer

Lexer is the main struct of the crate that allows you to read through a Source and produce tokens for enums implementing the Logos trait.

Skip

Type that can be returned from a callback, informing the Lexer, to skip current token match. See also logos::skip.

SpannedIter

Iterator that pairs tokens with their position in the source.

Enums

Filter

Type that can be returned from a callback, either producing a field for a token, or skipping it.

FilterResult

Type that can be returned from a callback, either producing a field for a token, skipping it, or emitting an error.

Traits

Logos

Trait implemented for an enum representing all tokens. You should never have to implement it manually, use the #[derive(Logos)] attribute on your enum.

Functions

skip

Predefined callback that will inform the Lexer to skip a definition.

Type Aliases

Span

Byte range in the source.

Derive Macros

Logosexport_derive