Wheelodex » Projects » tokenlens » tokenlens-0.1.6-py3-none-any.whl

tokenlens

View on PyPI — Reverse Dependencies (0)

0.1.6 tokenlens-0.1.6-py3-none-any.whl

Wheel Details

Project: tokenlens
Version: 0.1.6
Filename: tokenlens-0.1.6-py3-none-any.whl
Download: [link]
Size: 40074
MD5: b82b037ef2bd90dff8c3f7192862be49
SHA256: c24475d39884b67846d67ad1336eaaad9929181041fc2a59ff15dca76ee35f4c
Uploaded: 2025-01-05 03:40:18 +0000

dist-info

METADATA · WHEEL · RECORD · top_level.txt

METADATA

Metadata-Version: 2.1
Name: tokenlens
Version: 0.1.6
Summary: A library for accurate token counting and limit validation across various LLM providers
Author: TokenLens Team
Author-Email: Giri Ramanathan <vgiri2015[at]gmail.com>
Home-Page: https://github.com/tokenlens/tokenlens
Project-Url: Homepage, https://github.com/vgiri2015/tokenlens
Project-Url: Documentation, https://github.com/vgiri2015/tokenlens/tree/main/docs
Project-Url: Repository, https://github.com/vgiri2015/tokenlens.git
Project-Url: Bug Tracker, https://github.com/vgiri2015/tokenlens/issues
License: MIT
Keywords: llm,tokens,openai,mistral,anthropic,tokenizer
Classifier: Development Status :: 4 - Beta
Classifier: Intended Audience :: Developers
Classifier: License :: OSI Approved :: MIT License
Classifier: Programming Language :: Python :: 3
Classifier: Programming Language :: Python :: 3.8
Classifier: Programming Language :: Python :: 3.9
Classifier: Programming Language :: Python :: 3.10
Classifier: Programming Language :: Python :: 3.11
Classifier: Topic :: Scientific/Engineering :: Artificial Intelligence
Classifier: Topic :: Software Development :: Libraries :: Python Modules
Requires-Python: >=3.8
Requires-Dist: tiktoken (>=0.5.2)
Requires-Dist: pydantic (>=1.10.13)
Requires-Dist: python-dotenv (>=1.0.0)
Requires-Dist: PyYAML (>=6.0.1)
Requires-Dist: openai (>=1.3.7); extra == "openai"
Requires-Dist: anthropic (>=0.3.0); extra == "anthropic"
Requires-Dist: google-generativeai (>=0.3.0); extra == "google"
Requires-Dist: transformers (>=4.30.0); extra == "huggingface"
Requires-Dist: huggingface-hub (>=0.16.0); extra == "huggingface"
Requires-Dist: mistralai (>=0.0.7); extra == "mistral"
Requires-Dist: stability-sdk (>=0.8.0); extra == "stability"
Requires-Dist: torch (>=2.0.0); extra == "media"
Requires-Dist: Pillow (>=9.0.0); extra == "media"
Requires-Dist: numpy (>=1.21.0); extra == "media"
Requires-Dist: openai (>=1.3.7); extra == "all"
Requires-Dist: anthropic (>=0.3.0); extra == "all"
Requires-Dist: google-generativeai (>=0.3.0); extra == "all"
Requires-Dist: transformers (>=4.30.0); extra == "all"
Requires-Dist: huggingface-hub (>=0.16.0); extra == "all"
Requires-Dist: mistralai (>=0.0.7); extra == "all"
Requires-Dist: stability-sdk (>=0.8.0); extra == "all"
Requires-Dist: torch (>=2.0.0); extra == "all"
Requires-Dist: Pillow (>=9.0.0); extra == "all"
Requires-Dist: numpy (>=1.21.0); extra == "all"
Provides-Extra: openai
Provides-Extra: anthropic
Provides-Extra: google
Provides-Extra: huggingface
Provides-Extra: mistral
Provides-Extra: stability
Provides-Extra: media
Provides-Extra: all
Description-Content-Type: text/markdown
License-File: LICENSE
[Description omitted; length: 6213 characters]

WHEEL

Wheel-Version: 1.0
Generator: setuptools (75.6.0)
Root-Is-Purelib: true
Tag: py3-none-any

RECORD

Path Digest Size
tokenlens/__init__.py sha256=lRmHwhIVDd0AmW9pzIVgNFhtge-5rFTzQ8TBfXlX3gQ 1487
tokenlens/cli.py sha256=e2V0G4FK6rJPM4bQO2-xlDiNojWupyagjNqWWi-QYzo 527
tokenlens/tokenizer.py sha256=vUuewRv8ct8W2W1xpNbZoizTmCp5_iarUMN0rpZG6c4 1476
tokenlens/providers/__init__.py sha256=YktU7ivdhX4ykFgqPCNJSMCLA40Dj2n4FdvkiMbnlb8 5245
tokenlens/providers/adobe_provider.py sha256=I6aE4hPFdu0TRh0yCxu0eB_4dKJfh0jNtu_ZepfO2xU 2409
tokenlens/providers/ai21_provider.py sha256=5V4FGQsAqBV8V0xng5N_Beb5DxLBVh_1q_0WWytAsfg 2928
tokenlens/providers/amazon_provider.py sha256=o0ZY_4T4_ixXzwbDxbuA_N5ju1Dt9eaR-4cEJ4aWgjE 4083
tokenlens/providers/anthropic_provider.py sha256=wzjofHVLHbIDaACdXMeMUN-3mUxfghseNJHRy__5F_g 3302
tokenlens/providers/art_providers.py sha256=yE2tOxrGkR1FJgGSZFs17Iv3EFyKgrOyvgdIJEq8-yw 3304
tokenlens/providers/avatar_providers.py sha256=RTYI_DlkZSif8uGA-lcwmY9o9EM0aThdTFwsrEebBi0 2757
tokenlens/providers/cohere_provider.py sha256=wG2zPiWVMvXroTPSh97K_M10y4n9a0Fk8RMdEA1U7JI 4465
tokenlens/providers/deepmind_provider.py sha256=SBfUmloahLOl1A6NQTf1QuAQHFJXT67umQEzG-9ZX2k 1276
tokenlens/providers/factory.py sha256=WQ8jSBuK1pmMEfcFyksMbouAOpemi9JA5bYSCn-2jyw 2748
tokenlens/providers/gaming_providers.py sha256=5LDNcbzDVSQ_gfx7pnWq09SDopExfGXsaZw4VxlHd_E 1361
tokenlens/providers/google_provider.py sha256=m97BmustVCDAYEYlprPXcJthF-_BvI9YXqWgs2EArPs 3697
tokenlens/providers/haygen_provider.py sha256=Fee6xvjF3CdexgxXNf83FXqY9hWd_maIzJntKb-pido 4336
tokenlens/providers/huggingface_provider.py sha256=R-d9M60kccGEqDp3oKWcD2B9UDC48PTr9mtqeDj4a1s 1421
tokenlens/providers/meta_provider.py sha256=78_N3u5L7aKshdcfQhGCSD_00_59hZONiDSQT8VN8dE 4227
tokenlens/providers/microsoft_provider.py sha256=06puSKSrhWbJBECkkRd15pVfFHVQWvh6775r40qpbeM 1799
tokenlens/providers/midjourney_provider.py sha256=8if5nIqTRd4ku0h9qjnEN1HWLvGDV8fR4t9u2i1NJSE 4655
tokenlens/providers/mistral_provider.py sha256=X0l3pjwpTsCDkGoOig2EyJwLX5cCahkexHiYpcC1Co0 3666
tokenlens/providers/openai_provider.py sha256=q8_3Mm6Nkqqr2hE_y89IyZRLSZhazcb8uPGBlByqFPU 7826
tokenlens/providers/provider_factory.py sha256=EkkOzQ10EXZzPEQPMtEuG7nMs3VvQOegYHOBVHA2gsY 2141
tokenlens/providers/provider_template.py sha256=RLxh52juXvGhGGcytMnf8GwSGQWlJdVIr6Xnow5Up98 5747
tokenlens/providers/qwen_provider.py sha256=_MSqxAu5-HTNeJ0GA_PDAs6XSE2LveaVnNwDm3M_eoU 2657
tokenlens/providers/stability_provider.py sha256=IslOUlgMsaa83C6VsDzRYWGfQLwjm5v3f-1R0fmCL1Y 4705
tokenlens/providers/stanford_provider.py sha256=nIR-msmyfZaJhIzAfUiHugGSNAczHLwneAysFKyGqTI 1274
tokenlens/providers/voice_providers.py sha256=xmk1h7c6pwuOOp-TsJQ0c4jj5GkiYMk1gMxkCXk4DC8 1254
tokenlens/tokenizers/__init__.py sha256=u6zH2T0iE26sTABhj0nG5W91RvUYwNCt9CZSrrwrHxk 1666
tokenlens/tokenizers/ai21_tokenizer.py sha256=NicChXtVi1-t35eOeT9ZiNFCI_C4dy0v9TNtm4XnL2U 1493
tokenlens/tokenizers/anthropic_tokenizer.py sha256=_v5UHEx8t1J9zASeXdczDNIJegjt9GrJv2eYkjWjuVM 949
tokenlens/tokenizers/base.py sha256=0honjaoJ3yeWSsWvBo6RC4QkWTIHyknHeChzLF5bNBw 572
tokenlens/tokenizers/cohere_tokenizer.py sha256=XuxVLi6Qvygay0eWbs9ra5ZC7RPjpUh-wqTsp4ScERU 1448
tokenlens/tokenizers/deepmind_tokenizer.py sha256=FQvxm8abQMhLk4fsBcYNtYdUrdhM-h0Hvsd9ULX2Ymc 2304
tokenlens/tokenizers/factory.py sha256=dIvLFGWayYTrclWPysuAdOaSAle89g9Lo6cUTBlp-Rc 2761
tokenlens/tokenizers/google_tokenizer.py sha256=pk5I_CYhkhXARFItfd65qQnmHsPCL0b5kWxpDQZQLDM 1614
tokenlens/tokenizers/huggingface_tokenizer.py sha256=en9YwfSSodhihEjDk39v2WeVdwoY26DNovinDgjJ55Q 1696
tokenlens/tokenizers/meta_tokenizer.py sha256=QbLH0IedQ2mAL6HBg8YEDX3_T02Ig8DQDCE4FldRgd4 1898
tokenlens/tokenizers/mistral_tokenizer.py sha256=11JJuoaS7jhS_VQQfpWBdvQhGxmyDq5tah72BJc5Jjg 1503
tokenlens/tokenizers/openai_tokenizer.py sha256=3CI8xwc_WZpInFRT3atMaFWFq92XG76WLJiTApby6p0 1413
tokenlens/tokenizers/qwen_tokenizer.py sha256=-FQvanQi6axGi65_fQwyQwQXdSCMwFZvvOTrwOL1KqU 1702
tokenlens/tokenizers/stanford_tokenizer.py sha256=d6Hxr6Gmr4dyvbmuyMT2wt8A0v1JgjKmqnGG4pprJ1g 1713
tokenlens-0.1.6.dist-info/LICENSE sha256=TPg2yioNzqtq2Y16qbmMmm1LLUgEABsRZ_AcfZtEIOU 1072
tokenlens-0.1.6.dist-info/METADATA sha256=q66iUG0Tthh_jCwlaIyg9gtR6XB0XRBJt0vkhFOnYb0 8808
tokenlens-0.1.6.dist-info/WHEEL sha256=PZUExdf71Ui_so67QXpySuHtCi3-J3wvF4ORK6k_S8U 91
tokenlens-0.1.6.dist-info/top_level.txt sha256=F9XavUWX6wzEhKmWpeQlaUmcNFkhAN_Nr8YxHEAEFeM 10
tokenlens-0.1.6.dist-info/RECORD — —

top_level.txt

tokenlens