rz_tiktoken_ruby 0.0.6-aarch64-linux
An unofficial Ruby wrapper for Tiktoken, a BPE tokenizer written by and used by OpenAI. It can be used to count the number of tokens in text before sending it to OpenAI APIs. This is a fork of tiktoken_ruby by IAPark, which has been cross-compiled for multiple platforms. This way compilation with Rust extensions doesn't need to happen wherever you are deploying it.
Gemfile:
=
クリップボードにコピー
コピー完了!
インストール:
=
所有者:
プッシュ者:
作者:
-
IAPark, judy
SHA 256チェックサム:
35685e7dc1cc2f3471e3716ecefb1c74e3f5ed4596dd8a920f52e2add0f3afb5