Enum rustc_ap_rustc_lexer::TokenKind

pub enum TokenKind {
    LineComment {
        doc_style: Option<DocStyle>,
    },
    BlockComment {
        doc_style: Option<DocStyle>,
        terminated: bool,
    },
    Whitespace,
    Ident,
    RawIdent,
    Literal {
        kind: LiteralKind,
        suffix_start: usize,
    },
    Lifetime {
        starts_with_number: bool,
    },
    Semi,
    Comma,
    Dot,
    OpenParen,
    CloseParen,
    OpenBrace,
    CloseBrace,
    OpenBracket,
    CloseBracket,
    At,
    Pound,
    Tilde,
    Question,
    Colon,
    Dollar,
    Eq,
    Bang,
    Lt,
    Gt,
    Minus,
    And,
    Or,
    Plus,
    Star,
    Slash,
    Caret,
    Percent,
    Unknown,
}

Enum representing common lexeme types.

Variants

LineComment

"// comment"

Fields of LineComment

doc_style: Option<DocStyle>

BlockComment

/* block comment */

Block comments can be recursive, so the sequence like /* /* */ will not be considered terminated and will result in a parsing error.

Fields of BlockComment

doc_style: Option<DocStyle>terminated: bool

Whitespace

Any whitespace characters sequence.

Ident

"ident" or "continue" At this step keywords are also considered identifiers.

RawIdent

"r#ident"

Literal

"12_u8", "1.0e-40", "b"123"". See LiteralKind for more details.

Fields of Literal

kind: LiteralKindsuffix_start: usize

Lifetime

"'a"

Fields of Lifetime

starts_with_number: bool

Semi

";"

Comma

","

Dot

"."

OpenParen

"("

CloseParen

")"

OpenBrace

"{"

CloseBrace

"}"

OpenBracket

"["

CloseBracket

"]"

At

"@"

Pound

"#"

Tilde

"~"

Question

"?"

Colon

":"

Dollar

"$"

Eq

"="

Bang

"!"

Lt

"<"

Gt

">"

Minus

"-"

And

"&"

Or

"|"

Plus

"+"

Star

"*"

Slash

"/"

Caret

"^"

Percent

"%"

Unknown

Unknown token, not expected by the lexer, e.g. "№"

Trait Implementations

impl Clone for TokenKind

fn clone(&self) -> TokenKind

Returns a copy of the value.

fn clone_from(&mut self, source: &Self)

Performs copy-assignment from source.

impl Copy for TokenKind

impl Debug for TokenKind

fn fmt(&self, f: &mut Formatter<'_>) -> Result

Formats the value using the given formatter.

impl Eq for TokenKind

impl Ord for TokenKind

fn cmp(&self, other: &TokenKind) -> Ordering

This method returns an [Ordering] between self and other.

#[must_use]fn max(self, other: Self) -> Self

Compares and returns the maximum of two values.

#[must_use]fn min(self, other: Self) -> Self

Compares and returns the minimum of two values.

#[must_use]fn clamp(self, min: Self, max: Self) -> Self

🔬 This is a nightly-only experimental API. (clamp)

Restrict a value to a certain interval.

impl PartialEq<TokenKind> for TokenKind

fn eq(&self, other: &TokenKind) -> bool

This method tests for self and other values to be equal, and is used by ==.

fn ne(&self, other: &TokenKind) -> bool

This method tests for !=.

impl PartialOrd<TokenKind> for TokenKind

fn partial_cmp(&self, other: &TokenKind) -> Option<Ordering>

This method returns an ordering between self and other values if one exists.

fn lt(&self, other: &TokenKind) -> bool

This method tests less than (for self and other) and is used by the < operator.

fn le(&self, other: &TokenKind) -> bool

This method tests less than or equal to (for self and other) and is used by the <= operator.

fn gt(&self, other: &TokenKind) -> bool

This method tests greater than (for self and other) and is used by the > operator.

fn ge(&self, other: &TokenKind) -> bool

This method tests greater than or equal to (for self and other) and is used by the >= operator.

impl StructuralEq for TokenKind

impl StructuralPartialEq for TokenKind