Crate msiz_rustc_ap_rustc_lexer

Modules

• unescape
  Utilities for validating string and char literals and turning them into values they represent.

Structs

• Token

Enums

• Base
• LiteralKind
• TokenKind

Functions

• first_token
• is_id_continue
  True if c is valid as a non-first character of an identifier.
• is_id_start
  True if c is valid as a first character of an identifier.
• is_whitespace
  True if c is considered a whitespace according to Rust language definition.
• strip_shebang
• tokenize