tokenizers 0.1.1
Fast state-of-the-art tokenizers for Ruby
Gemfile:
=
Copy to clipboard
Copied!
install:
=
Versions:
- 0.4.4 - February 27, 2024 x86_64-linux (7.5 MB)
- 0.4.4 - February 27, 2024 x86_64-linux-musl (7.51 MB)
- 0.4.4 - February 27, 2024 x86_64-darwin (7.82 MB)
- 0.4.4 - February 27, 2024 arm64-darwin (7.45 MB)
- 0.4.4 - February 27, 2024 aarch64-linux (7 MB)
- 0.1.1 - June 29, 2022 (24 KB)