Home
Packages
Datasets
Task Views
R resources
Packages
Toggle theme
Toggle Menu
Home
Packages
rtiktoken
get_tokens
get_tokens function
Converts text to tokens
Copy
rtiktoken package
Read PDF manual
Maintainer: David Zimmermann-Kollenda
License: MIT + file LICENSE
Last published: 2025-04-14
Useful links
https://github.com/DavZim/rtiktoken/issues
https://davzim.github.io/rtiktoken/
https://github.com/DavZim/rtiktoken/