Wheelodex » Projects » llm-optimized-inference

llm-optimized-inference

View on PyPI — Reverse Dependencies (0)

0.2.17 llm_optimized_inference-0.2.17-py3-none-any.whl

Wheel Details

Project: llm-optimized-inference
Version: 0.2.17
Filename: llm_optimized_inference-0.2.17-py3-none-any.whl
Download: [link]
Size: 84180
MD5: d4b07ee9bc106822eb9d9047319220a2
SHA256: f190377b8aa3a0207458eda62ca5557909538ee6e30ce19c669ff6c971823ce6
Uploaded: 2025-01-28 21:40:05 +0000

dist-info

METADATA · WHEEL · RECORD · top_level.txt

METADATA

Metadata-Version: 2.2
Name: llm-optimized-inference
Version: 0.2.17
Author: Microsoft
Classifier: Development Status :: 3 - Alpha
Classifier: Intended Audience :: Developers
Classifier: License :: OSI Approved :: MIT License
Classifier: Programming Language :: Python :: 3.9
Classifier: Programming Language :: Python :: 3.10
Requires-Python: >=3.9
Requires-Dist: deepspeed (==0.15.1)
Requires-Dist: vllm (==0.6.3.post1)
Requires-Dist: deepspeed-kernels (==0.0.1.dev1698255861)
Requires-Dist: diffusers (==0.26.2)
Requires-Dist: pandas (~=2.1.4)
Requires-Dist: transformers (==4.46.2)
Requires-Dist: aiolimiter (~=1.1.0)
Requires-Dist: azure-ai-contentsafety (==1.0.0b1)
Requires-Dist: azure-ai-ml (==1.12.1)
Requires-Dist: azure-identity
Requires-Dist: azureml-ai-monitoring (==0.1.0b4)
Requires-Dist: azureml-inference-server-http
Requires-Dist: azureml-mlflow
Requires-Dist: requests (~=2.32.0)
Requires-Dist: aiohttp (~=3.10.0)
Requires-Dist: torch (~=2.4.0)
Requires-Dist: scipy
Requires-Dist: accelerate (>=0.20.3)
Requires-Dist: sacremoses
Requires-Dist: fastapi (==0.112.4)
Requires-Dist: Jinja2 (>=3.1.4)
Requires-Dist: Flask-Cors (==5.0.0)
Requires-Dist: gunicorn (>=23.0.0)
Requires-Dist: deepspeed-mii (==0.3.0)
Requires-Dist: soundfile (>=0.13.0)
Requires-Dist: scipy (>=1.15.1)
Requires-Dist: backoff (>=2.2.1)
Requires-Dist: flash-attn (>=2.7.3)
Requires-Dist: librosa (>=0.10.2.post1)
Requires-Dist: pytest; extra == "dev"
Provides-Extra: dev
Description-Content-Type: text/markdown
Dynamic: author
Dynamic: classifier
Dynamic: description
Dynamic: description-content-type
Dynamic: provides-extra
Dynamic: requires-dist
Dynamic: requires-python
License-File: LICENSE
[Description omitted; length: 1927 characters]

WHEEL

Wheel-Version: 1.0
Generator: setuptools (75.8.0)
Root-Is-Purelib: true
Tag: py3-none-any

RECORD

Path Digest Size
llm/__init__.py sha256=ybCtCrXNS3six25SvldGKjcl-9eR9CzOMgoUjbRBdBQ 249
llm/optimized/__init__.py sha256=ybCtCrXNS3six25SvldGKjcl-9eR9CzOMgoUjbRBdBQ 249
llm/optimized/inference/__init__.py sha256=rGZODP-qE2NOlUgdbmxhfuS3svmx9gM-kU97d9-jb50 703
llm/optimized/inference/_version.py sha256=75jJyVV1Am6IWq0bMn-ovJPbB_Nigti0SdjQQTUPFNE 205
llm/optimized/inference/api_server.py sha256=Urzl_yBBGilHnHZCBy4nY5SN44J5cq4us8WX2bkgmCg 38250
llm/optimized/inference/configs.py sha256=-MfjhLAjdLbCSJFrk5cjjzZGG3dmmJ2HnTlON6r4I68 3327
llm/optimized/inference/constants.py sha256=ZpAg6XB9BpkDk4Q46fPZ1Fe7xMHE14sKQedBwLQU_b0 15993
llm/optimized/inference/conversation.py sha256=cBrHv1sI1hrOQB57sSZKAarplKRgqQWqmYxT-wkZHLM 3255
llm/optimized/inference/error_handler.py sha256=UN25QkbbU9U6SGtTV-sFQ3Pzos35h2EDx2n5_O2-A0c 1457
llm/optimized/inference/fm_score.py sha256=sh5n4LtS_RsDgtoR8sQN-FC6bHT4Vu6L5GKTBkh07SA 6035
llm/optimized/inference/interceptor.py sha256=-Ecf3kTNqu6cwX8vn5vBmnSXQ5882bjsU8V4dpLpr_w 3152
llm/optimized/inference/logging_config.py sha256=egtAiGCNVNoU8E_JPhp2aGJVIJ_QkTwLhkNjpVKjICI 664
llm/optimized/inference/managed_inference.py sha256=2eIsX5QXHynhQhKpt--R1BcuhvIoFMMCuwHWgUPBWWs 9503
llm/optimized/inference/model_config_factory.py sha256=EpHH8QyUaE5DC6A5O5nKbie_kMEpfeqmufbOn0U6jFI 1369
llm/optimized/inference/model_utils.py sha256=w3XZanJKKJ82IoHlo1H4caxCP-9ijI27HSWFRyD_TCM 10142
llm/optimized/inference/prompt_formatter.py sha256=xxM4MbvvPL6jQcLB37uKA9-wCHXTZUL9l5GiUu3r964 1575
llm/optimized/inference/replica_manager.py sha256=Oir7fibeFekUNDldeEL_l_1oopOH8BQSUIa5qgX_9Es 13762
llm/optimized/inference/request_adapter.py sha256=SXcd8mlXZVZP6gWkXPuFQeBkZ5kieJgk9JIBX99ojK8 5486
llm/optimized/inference/score.py sha256=RWEaCZ4Akig88ErqshDD8KFCF-YDwMJvsRJjeBWR-WM 18786
llm/optimized/inference/utils.py sha256=GYUA9EKWZ-2pZD0qV05VKcppLVTK2yy9SG25_KIfFMM 4146
llm/optimized/inference/vllm_api_server.py sha256=eHRTOVoEokIZ-DEvejjvMT5nJ13ukr8Fam6RQGP1Oug 24793
llm/optimized/inference/api_server_setup/__init__.py sha256=47DEQpj8HBSa-_TImW-5JCeuQeRkm5NMpJWZG3hSuFU 0
llm/optimized/inference/api_server_setup/openapi.json sha256=V1vPqe36Sh-WpDLukfVk96iXut3AEI8lLTKWXCcQEWU 9703
llm/optimized/inference/api_server_setup/protocol.py sha256=wZ0FB7ZkyPmKtrvdTyPFwVv7HCtvaxJT5nmUlQAvz_M 15441
llm/optimized/inference/custom_model_configurations/__init__.py sha256=D0zlB3WPH212MfcHh1Wz6TRe6Igcg-0Wh0hHA8Kl_l0 131
llm/optimized/inference/custom_model_configurations/base_configuration_builder.py sha256=pX9BNG70Rts3TY5p_Smuq4x8ZDG0DG1-6kLfY90Jht0 2725
llm/optimized/inference/custom_model_configurations/diffusion_configuration_builder.py sha256=oPY_h5WDdopurQzsfaR95DHLjgFSNgq4coE2ETo-j2o 8806
llm/optimized/inference/custom_model_configurations/schema_output.py sha256=JIpThLkAPoO7MlhaJVSxP2RYVwYzIpw3R3k-nraLq9k 1047
llm/optimized/inference/engine/__init__.py sha256=8GqGW53dbRlGShwO11h5HQH4KjnUzoW0t0cChUUFDPY 259
llm/optimized/inference/engine/_hf_predictors.py sha256=2tUoPPy61xzL4zzpONAliT4a9TBakjTCcTb8osz1BlE 34382
llm/optimized/inference/engine/engine.py sha256=POUIcp_Jx3Zue5zqWtMRC4iJaK_xawTcOu8x3GbS3sM 4920
llm/optimized/inference/engine/hf_engine.py sha256=xwPRw3w7-IBLfqnuKJcRmWDYH9we4YNNSRKoe7zp3_g 8852
llm/optimized/inference/engine/mii_engine.py sha256=OG-F5oGgLxvXUMExjCvqf6ieD3Yzrnbsug49Q-wjUqw 7222
llm/optimized/inference/engine/mii_engine_v2.py sha256=yz7aru0NezlRBO8RafiVTcZ3RWQ_92QjcbPBIAf6jx4 6043
llm/optimized/inference/engine/vllm_engine.py sha256=36xxan-_n96FnRqFcr5GsU0nSP3h2OH5ijpAeWlIdFg 19279
llm_optimized_inference-0.2.17.dist-info/LICENSE sha256=47DEQpj8HBSa-_TImW-5JCeuQeRkm5NMpJWZG3hSuFU 0
llm_optimized_inference-0.2.17.dist-info/METADATA sha256=ZtvqbQVPFV6EJJm7TENdRSPg3AkObXg2TWGNpZ0L11w 3567
llm_optimized_inference-0.2.17.dist-info/WHEEL sha256=In9FTNxeP60KnTkGw7wk6mJPYd_dQSjEZmXdBdMCI-8 91
llm_optimized_inference-0.2.17.dist-info/top_level.txt sha256=TwpEQXP3b1MS9Y2XuGgRuo9-Kny507xt2HFZgJ5TSIY 4
llm_optimized_inference-0.2.17.dist-info/RECORD — —

top_level.txt

llm