Function cedar_policy_formatter::lexer::get_token_stream
source ยท pub fn get_token_stream(input: &str) -> Option<(Vec<WrappedToken>, String)>
Expand description
Tokenize the input, associating with each token a leading and trailing comment if they are present. Also returns a string containing any comments that may be present at the end of input after all tokens are consumed.