Wheelodex » Projects » llm-optimized-inference

llm-optimized-inference

View on PyPI — Reverse Dependencies (0)

0.2.21 llm_optimized_inference-0.2.21-py3-none-any.whl

Wheel Details

Project: llm-optimized-inference
Version: 0.2.21
Filename: llm_optimized_inference-0.2.21-py3-none-any.whl
Download: [link]
Size: 85635
MD5: b0954c9e72d5807a9c0c5b1aa1ba832d
SHA256: 3f51d32dbda840f4a74f95243c2baebfd3442a0c056dfd01c7a7caed05809f98
Uploaded: 2025-03-04 15:26:59 +0000

dist-info

METADATA · WHEEL · RECORD · top_level.txt

METADATA

Metadata-Version: 2.2
Name: llm-optimized-inference
Version: 0.2.21
Author: Microsoft
Classifier: Development Status :: 3 - Alpha
Classifier: Intended Audience :: Developers
Classifier: License :: OSI Approved :: MIT License
Classifier: Programming Language :: Python :: 3.9
Classifier: Programming Language :: Python :: 3.10
Requires-Python: >=3.9
Requires-Dist: deepspeed (==0.15.1)
Requires-Dist: vllm (==0.7.3)
Requires-Dist: deepspeed-kernels (==0.0.1.dev1698255861)
Requires-Dist: diffusers (==0.26.2)
Requires-Dist: pandas (~=2.1.4)
Requires-Dist: transformers (==4.48.2)
Requires-Dist: aiolimiter (~=1.1.0)
Requires-Dist: azure-ai-contentsafety (==1.0.0b1)
Requires-Dist: azure-ai-ml (==1.12.1)
Requires-Dist: azure-identity
Requires-Dist: azureml-ai-monitoring (==0.1.0b4)
Requires-Dist: azureml-inference-server-http
Requires-Dist: azureml-mlflow
Requires-Dist: requests (~=2.32.0)
Requires-Dist: aiohttp (~=3.10.0)
Requires-Dist: torch (~=2.5.1)
Requires-Dist: scipy
Requires-Dist: accelerate (>=0.20.3)
Requires-Dist: sacremoses
Requires-Dist: fastapi (==0.112.4)
Requires-Dist: Jinja2 (>=3.1.4)
Requires-Dist: Flask-Cors (==5.0.0)
Requires-Dist: gunicorn (>=23.0.0)
Requires-Dist: deepspeed-mii (==0.3.0)
Requires-Dist: soundfile (>=0.13.0)
Requires-Dist: scipy (>=1.15.1)
Requires-Dist: backoff (>=2.2.1)
Requires-Dist: flash-attn (>=2.7.3)
Requires-Dist: librosa (>=0.10.2.post1)
Requires-Dist: pytest; extra == "dev"
Provides-Extra: dev
Description-Content-Type: text/markdown
Dynamic: author
Dynamic: classifier
Dynamic: description
Dynamic: description-content-type
Dynamic: provides-extra
Dynamic: requires-dist
Dynamic: requires-python
License-File: LICENSE
[Description omitted; length: 1927 characters]

WHEEL

Wheel-Version: 1.0
Generator: setuptools (75.8.2)
Root-Is-Purelib: true
Tag: py3-none-any

RECORD

Path Digest Size
llm/__init__.py sha256=1CkzgkzHSUZyj73EBCf5Wm0-x4YtYICnOD94xr_hK8Q 2232
llm/optimized/__init__.py sha256=ybCtCrXNS3six25SvldGKjcl-9eR9CzOMgoUjbRBdBQ 249
llm/optimized/inference/__init__.py sha256=rGZODP-qE2NOlUgdbmxhfuS3svmx9gM-kU97d9-jb50 703
llm/optimized/inference/_version.py sha256=sTcRhcFMDqpliIBl6BIw9VRGB9wwX2rTRXjgROMK4Bs 205
llm/optimized/inference/api_server.py sha256=Urzl_yBBGilHnHZCBy4nY5SN44J5cq4us8WX2bkgmCg 38250
llm/optimized/inference/configs.py sha256=-MfjhLAjdLbCSJFrk5cjjzZGG3dmmJ2HnTlON6r4I68 3327
llm/optimized/inference/constants.py sha256=ZpAg6XB9BpkDk4Q46fPZ1Fe7xMHE14sKQedBwLQU_b0 15993
llm/optimized/inference/conversation.py sha256=vif4t7-b18d6Qn85N6I5Up6Plh55WiHE5iHwsbZiJ4s 4615
llm/optimized/inference/error_handler.py sha256=UN25QkbbU9U6SGtTV-sFQ3Pzos35h2EDx2n5_O2-A0c 1457
llm/optimized/inference/fm_score.py sha256=sh5n4LtS_RsDgtoR8sQN-FC6bHT4Vu6L5GKTBkh07SA 6035
llm/optimized/inference/interceptor.py sha256=-Ecf3kTNqu6cwX8vn5vBmnSXQ5882bjsU8V4dpLpr_w 3152
llm/optimized/inference/logging_config.py sha256=egtAiGCNVNoU8E_JPhp2aGJVIJ_QkTwLhkNjpVKjICI 664
llm/optimized/inference/managed_inference.py sha256=FokAAo8izZfDIo2_dKZe3rjbXg6KJOqfuZdHKL_Uw4A 10117
llm/optimized/inference/model_config_factory.py sha256=EpHH8QyUaE5DC6A5O5nKbie_kMEpfeqmufbOn0U6jFI 1369
llm/optimized/inference/model_utils.py sha256=_a86f_HDvo-LB5rIANYywUlz5-2l-jzfLAwv6x8VQkM 10328
llm/optimized/inference/prompt_formatter.py sha256=xxM4MbvvPL6jQcLB37uKA9-wCHXTZUL9l5GiUu3r964 1575
llm/optimized/inference/replica_manager.py sha256=9TdOGLRO1v0ARA77jztlRJV5F3BK7oOeha8tf2apMkI 14015
llm/optimized/inference/request_adapter.py sha256=SXcd8mlXZVZP6gWkXPuFQeBkZ5kieJgk9JIBX99ojK8 5486
llm/optimized/inference/score.py sha256=RWEaCZ4Akig88ErqshDD8KFCF-YDwMJvsRJjeBWR-WM 18786
llm/optimized/inference/utils.py sha256=GYUA9EKWZ-2pZD0qV05VKcppLVTK2yy9SG25_KIfFMM 4146
llm/optimized/inference/vllm_api_server.py sha256=JREFVSihC1uew45b43eoNUEoeYLaUEpkZPx8MPz17jk 25556
llm/optimized/inference/api_server_setup/__init__.py sha256=47DEQpj8HBSa-_TImW-5JCeuQeRkm5NMpJWZG3hSuFU 0
llm/optimized/inference/api_server_setup/openapi.json sha256=V1vPqe36Sh-WpDLukfVk96iXut3AEI8lLTKWXCcQEWU 9703
llm/optimized/inference/api_server_setup/protocol.py sha256=wZ0FB7ZkyPmKtrvdTyPFwVv7HCtvaxJT5nmUlQAvz_M 15441
llm/optimized/inference/custom_model_configurations/__init__.py sha256=D0zlB3WPH212MfcHh1Wz6TRe6Igcg-0Wh0hHA8Kl_l0 131
llm/optimized/inference/custom_model_configurations/base_configuration_builder.py sha256=pX9BNG70Rts3TY5p_Smuq4x8ZDG0DG1-6kLfY90Jht0 2725
llm/optimized/inference/custom_model_configurations/diffusion_configuration_builder.py sha256=oPY_h5WDdopurQzsfaR95DHLjgFSNgq4coE2ETo-j2o 8806
llm/optimized/inference/custom_model_configurations/schema_output.py sha256=JIpThLkAPoO7MlhaJVSxP2RYVwYzIpw3R3k-nraLq9k 1047
llm/optimized/inference/engine/__init__.py sha256=8GqGW53dbRlGShwO11h5HQH4KjnUzoW0t0cChUUFDPY 259
llm/optimized/inference/engine/_hf_predictors.py sha256=oRk5olmU9QDl_xuYUbaHJJUHb5GVNUWqlnHDnwvzLSs 34253
llm/optimized/inference/engine/engine.py sha256=POUIcp_Jx3Zue5zqWtMRC4iJaK_xawTcOu8x3GbS3sM 4920
llm/optimized/inference/engine/hf_engine.py sha256=iSejUa2CRKgN_3iubPTskeOjK0kQQyTfT9Zcp6J2MTI 8873
llm/optimized/inference/engine/mii_engine.py sha256=OG-F5oGgLxvXUMExjCvqf6ieD3Yzrnbsug49Q-wjUqw 7222
llm/optimized/inference/engine/mii_engine_v2.py sha256=yz7aru0NezlRBO8RafiVTcZ3RWQ_92QjcbPBIAf6jx4 6043
llm/optimized/inference/engine/vllm_engine.py sha256=36xxan-_n96FnRqFcr5GsU0nSP3h2OH5ijpAeWlIdFg 19279
llm_optimized_inference-0.2.21.dist-info/LICENSE sha256=47DEQpj8HBSa-_TImW-5JCeuQeRkm5NMpJWZG3hSuFU 0
llm_optimized_inference-0.2.21.dist-info/METADATA sha256=B6pTbKGtzbP2W0RlA0_5Y81aIhAFkCcWu7HKvpTGawQ 3561
llm_optimized_inference-0.2.21.dist-info/WHEEL sha256=jB7zZ3N9hIM9adW7qlTAyycLYW9npaWKLRzaoVcLKcM 91
llm_optimized_inference-0.2.21.dist-info/top_level.txt sha256=TwpEQXP3b1MS9Y2XuGgRuo9-Kny507xt2HFZgJ5TSIY 4
llm_optimized_inference-0.2.21.dist-info/RECORD — —

top_level.txt

llm