Wheelodex » Projects » llm-optimized-inference

llm-optimized-inference

View on PyPI — Reverse Dependencies (0)

0.2.16 llm_optimized_inference-0.2.16-py3-none-any.whl

Wheel Details

Project: llm-optimized-inference
Version: 0.2.16
Filename: llm_optimized_inference-0.2.16-py3-none-any.whl
Download: [link]
Size: 82131
MD5: 5db574672b7eeca9abf8f89186d9c77e
SHA256: 6a283b37fc98bc9b60cea7597069f02c40c79909cb91056d9334da6ba53fccc2
Uploaded: 2024-10-30 03:18:03 +0000

dist-info

METADATA · WHEEL · RECORD · top_level.txt

METADATA

Metadata-Version: 2.1
Name: llm-optimized-inference
Version: 0.2.16
Author: Microsoft
Classifier: Development Status :: 3 - Alpha
Classifier: Intended Audience :: Developers
Classifier: License :: OSI Approved :: MIT License
Classifier: Programming Language :: Python :: 3.9
Classifier: Programming Language :: Python :: 3.10
Requires-Python: >=3.9
Requires-Dist: deepspeed (==0.15.1)
Requires-Dist: vllm (==0.6.1)
Requires-Dist: deepspeed-kernels (==0.0.1.dev1698255861)
Requires-Dist: diffusers (==0.26.2)
Requires-Dist: pandas (~=2.1.4)
Requires-Dist: transformers (~=4.43.2)
Requires-Dist: aiolimiter (~=1.1.0)
Requires-Dist: azure-ai-contentsafety (==1.0.0b1)
Requires-Dist: azure-ai-ml (==1.12.1)
Requires-Dist: azure-identity
Requires-Dist: azureml-ai-monitoring (==0.1.0b4)
Requires-Dist: azureml-inference-server-http
Requires-Dist: azureml-mlflow
Requires-Dist: requests (~=2.32.0)
Requires-Dist: aiohttp (~=3.10.0)
Requires-Dist: torch (~=2.4.0)
Requires-Dist: scipy
Requires-Dist: accelerate (>=0.20.3)
Requires-Dist: sacremoses
Requires-Dist: fastapi (==0.112.4)
Requires-Dist: Jinja2 (>=3.1.4)
Requires-Dist: Flask-Cors (==5.0.0)
Requires-Dist: gunicorn (>=23.0.0)
Requires-Dist: deepspeed-mii (==0.3.0)
Requires-Dist: pytest; extra == "dev"
Provides-Extra: dev
Description-Content-Type: text/markdown
License-File: LICENSE
[Description omitted; length: 1927 characters]

WHEEL

Wheel-Version: 1.0
Generator: setuptools (75.2.0)
Root-Is-Purelib: true
Tag: py3-none-any

RECORD

Path Digest Size
llm/__init__.py sha256=ybCtCrXNS3six25SvldGKjcl-9eR9CzOMgoUjbRBdBQ 249
llm/optimized/__init__.py sha256=ybCtCrXNS3six25SvldGKjcl-9eR9CzOMgoUjbRBdBQ 249
llm/optimized/inference/__init__.py sha256=rGZODP-qE2NOlUgdbmxhfuS3svmx9gM-kU97d9-jb50 703
llm/optimized/inference/_version.py sha256=TtuxZyLLtcDvqag0cqJXEyd6XCJCZUIwfScDSNp_HCg 205
llm/optimized/inference/api_server.py sha256=ic1tWI-7cTD-5rJAftwMYuJ03iRPDKxY2qswYpY2WAE 38274
llm/optimized/inference/configs.py sha256=-MfjhLAjdLbCSJFrk5cjjzZGG3dmmJ2HnTlON6r4I68 3327
llm/optimized/inference/constants.py sha256=adetsX8N_X_oxl6dJkISikaSMBxc5Zl01QjKbn1RNho 15629
llm/optimized/inference/conversation.py sha256=cBrHv1sI1hrOQB57sSZKAarplKRgqQWqmYxT-wkZHLM 3255
llm/optimized/inference/error_handler.py sha256=UN25QkbbU9U6SGtTV-sFQ3Pzos35h2EDx2n5_O2-A0c 1457
llm/optimized/inference/fm_score.py sha256=sh5n4LtS_RsDgtoR8sQN-FC6bHT4Vu6L5GKTBkh07SA 6035
llm/optimized/inference/logging_config.py sha256=egtAiGCNVNoU8E_JPhp2aGJVIJ_QkTwLhkNjpVKjICI 664
llm/optimized/inference/managed_inference.py sha256=2RuExRc3zUzFpAH-NuVjRPDAzMcx1yaucMo4r-rd7LI 9478
llm/optimized/inference/model_config_factory.py sha256=EpHH8QyUaE5DC6A5O5nKbie_kMEpfeqmufbOn0U6jFI 1369
llm/optimized/inference/model_utils.py sha256=Af-KgfvtnEiUEKzIE-6NVo_atfpB6TIhEKU4logAW0M 9529
llm/optimized/inference/prompt_formatter.py sha256=xxM4MbvvPL6jQcLB37uKA9-wCHXTZUL9l5GiUu3r964 1575
llm/optimized/inference/replica_manager.py sha256=Oir7fibeFekUNDldeEL_l_1oopOH8BQSUIa5qgX_9Es 13762
llm/optimized/inference/request_adapter.py sha256=SXcd8mlXZVZP6gWkXPuFQeBkZ5kieJgk9JIBX99ojK8 5486
llm/optimized/inference/score.py sha256=RWEaCZ4Akig88ErqshDD8KFCF-YDwMJvsRJjeBWR-WM 18786
llm/optimized/inference/utils.py sha256=GYUA9EKWZ-2pZD0qV05VKcppLVTK2yy9SG25_KIfFMM 4146
llm/optimized/inference/vllm_api_server.py sha256=cWXQhvS3SGL-ijCaomltyPfdUw6K8394GL-En1OxOJ8 24200
llm/optimized/inference/api_server_setup/__init__.py sha256=47DEQpj8HBSa-_TImW-5JCeuQeRkm5NMpJWZG3hSuFU 0
llm/optimized/inference/api_server_setup/openapi.json sha256=V1vPqe36Sh-WpDLukfVk96iXut3AEI8lLTKWXCcQEWU 9703
llm/optimized/inference/api_server_setup/protocol.py sha256=TGvezfn7uo4uM6cTOh7SftSpdIgelKJHLJhF3C10wdc 14491
llm/optimized/inference/custom_model_configurations/__init__.py sha256=D0zlB3WPH212MfcHh1Wz6TRe6Igcg-0Wh0hHA8Kl_l0 131
llm/optimized/inference/custom_model_configurations/base_configuration_builder.py sha256=pX9BNG70Rts3TY5p_Smuq4x8ZDG0DG1-6kLfY90Jht0 2725
llm/optimized/inference/custom_model_configurations/diffusion_configuration_builder.py sha256=oPY_h5WDdopurQzsfaR95DHLjgFSNgq4coE2ETo-j2o 8806
llm/optimized/inference/custom_model_configurations/schema_output.py sha256=JIpThLkAPoO7MlhaJVSxP2RYVwYzIpw3R3k-nraLq9k 1047
llm/optimized/inference/engine/__init__.py sha256=8GqGW53dbRlGShwO11h5HQH4KjnUzoW0t0cChUUFDPY 259
llm/optimized/inference/engine/_hf_predictors.py sha256=2tUoPPy61xzL4zzpONAliT4a9TBakjTCcTb8osz1BlE 34382
llm/optimized/inference/engine/engine.py sha256=POUIcp_Jx3Zue5zqWtMRC4iJaK_xawTcOu8x3GbS3sM 4920
llm/optimized/inference/engine/hf_engine.py sha256=xwPRw3w7-IBLfqnuKJcRmWDYH9we4YNNSRKoe7zp3_g 8852
llm/optimized/inference/engine/mii_engine.py sha256=OG-F5oGgLxvXUMExjCvqf6ieD3Yzrnbsug49Q-wjUqw 7222
llm/optimized/inference/engine/mii_engine_v2.py sha256=yz7aru0NezlRBO8RafiVTcZ3RWQ_92QjcbPBIAf6jx4 6043
llm/optimized/inference/engine/vllm_engine.py sha256=REwVOTmFwkP7HRhjBGoJYs0kpRsshS-TkVgKyNk1_Mg 18081
llm_optimized_inference-0.2.16.dist-info/LICENSE sha256=47DEQpj8HBSa-_TImW-5JCeuQeRkm5NMpJWZG3hSuFU 0
llm_optimized_inference-0.2.16.dist-info/METADATA sha256=usYWTIqqHlfKshoraP91fJ5qGhTgLSb9jlvyD9rmkSI 3236
llm_optimized_inference-0.2.16.dist-info/WHEEL sha256=OVMc5UfuAQiSplgO0_WdW7vXVGAt9Hdd6qtN4HotdyA 91
llm_optimized_inference-0.2.16.dist-info/top_level.txt sha256=TwpEQXP3b1MS9Y2XuGgRuo9-Kny507xt2HFZgJ5TSIY 4
llm_optimized_inference-0.2.16.dist-info/RECORD — —

top_level.txt

llm