rustc-ap-rustc_lexer 657.0.0

Automatically published version of the package `rustc_lexer` in the rust-lang/rust repository from commit 04689e22e946879f2f5e2c73849d2f4e1f2b4b32 The publishing script for this crate lives at: https://github.com/alexcrichton/rustc-auto-publish
Documentation

Low-level Rust lexer.

Tokens produced by this lexer are not yet ready for parsing the Rust syntax, for that see librustc_parse::lexer, which converts this basic token stream into wide tokens used by actual parser.

The purpose of this crate is to convert raw sources into a labeled sequence of well-known token types, so building an actual Rust token stream will be easier.

Main entity of this crate is [TokenKind] enum which represents common lexeme types.