tokenize function

Tokenize a character vector Parse the elements of a character vector into a list of cleaned tokens.