pub fn get_token_stream(input: &str) -> Option<(Vec<WrappedToken>, String)>Expand description
Tokenize the input, associating with each token a leading and trailing comment if they are present. Also returns a string containing any comments that may be present at the end of input after all tokens are consumed.