Wheelodex » Projects » lighteval

lighteval

View on PyPI — Reverse Dependencies (2)

0.7.0 lighteval-0.7.0-py3-none-any.whl

Wheel Details

Project: lighteval
Version: 0.7.0
Filename: lighteval-0.7.0-py3-none-any.whl
Download: [link]
Size: 351006
MD5: 6893f6ecb057ee2afc8ad9967b56117e
SHA256: 2821ff1cda1bfe387bf90bc88cdc6032192f5038ef3802dce5398d1e67a3daa8
Uploaded: 2025-01-03 15:44:52 +0000

dist-info

METADATA · WHEEL · RECORD · top_level.txt · entry_points.txt

METADATA

Metadata-Version: 2.1
Name: lighteval
Version: 0.7.0
Summary: A lightweight and configurable evaluation package
Author-Email: Clémentine Fourrier <clementine[at]huggingface.com>, Nathan Habib <nathan.habib[at]huggingface.com>, Thomas Wolf <thom[at]huggingface.com>
Maintainer-Email: Clémentine Fourrier <clementine[at]huggingface.com>, Nathan Habib <nathan.habib[at]huggingface.com>
Project-Url: Homepage, https://github.com/huggingface/lighteval
Project-Url: Issues, https://github.com/huggingface/lighteval/issues
License: MIT License
Keywords: evaluation,nlp,llm
Classifier: Development Status :: 3 - Alpha
Classifier: Intended Audience :: Developers
Classifier: Programming Language :: Python :: 3.10
Classifier: License :: OSI Approved :: MIT License
Classifier: Operating System :: OS Independent
Requires-Python: >=3.10
Requires-Dist: transformers (>=4.38.0)
Requires-Dist: accelerate
Requires-Dist: huggingface_hub (>=0.23.0)
Requires-Dist: torch (<2.5,>=2.0)
Requires-Dist: GitPython (>=3.1.41)
Requires-Dist: datasets (>=2.14.0)
Requires-Dist: numpy (<2)
Requires-Dist: typer
Requires-Dist: termcolor (==2.3.0)
Requires-Dist: pytablewriter
Requires-Dist: rich
Requires-Dist: colorlog
Requires-Dist: aenum (==3.1.15)
Requires-Dist: nltk (==3.9.1)
Requires-Dist: scikit-learn
Requires-Dist: spacy (==3.7.2)
Requires-Dist: sacrebleu
Requires-Dist: rouge_score (==0.1.2)
Requires-Dist: sentencepiece (>=0.1.99)
Requires-Dist: protobuf (==3.20.*)
Requires-Dist: pycountry
Requires-Dist: fsspec (>=2023.12.2)
Requires-Dist: litellm; extra == "litellm"
Requires-Dist: diskcache; extra == "litellm"
Requires-Dist: text-generation (==0.6.0); extra == "tgi"
Requires-Dist: optimum (==1.12.0); extra == "optimum"
Requires-Dist: bitsandbytes (>=0.41.0); extra == "quantization"
Requires-Dist: auto-gptq (>=0.4.2); extra == "quantization"
Requires-Dist: peft (==0.3.0); extra == "adapters"
Requires-Dist: nanotron; extra == "nanotron"
Requires-Dist: tensorboardX; extra == "nanotron"
Requires-Dist: tensorboardX; extra == "tensorboardx"
Requires-Dist: vllm; extra == "vllm"
Requires-Dist: ray; extra == "vllm"
Requires-Dist: more_itertools; extra == "vllm"
Requires-Dist: ruff (==v0.2.2); extra == "quality"
Requires-Dist: pre-commit; extra == "quality"
Requires-Dist: pytest (==7.4.0); extra == "tests"
Requires-Dist: lighteval[accelerate,multilingual,quality,tests]; extra == "dev"
Requires-Dist: hf-doc-builder; extra == "docs"
Requires-Dist: watchdog; extra == "docs"
Requires-Dist: langdetect; extra == "extended-tasks"
Requires-Dist: openai; extra == "extended-tasks"
Requires-Dist: tiktoken; extra == "extended-tasks"
Requires-Dist: s3fs; extra == "s3"
Requires-Dist: stanza; extra == "multilingual"
Requires-Dist: spacy[ja,ko,th]; extra == "multilingual"
Requires-Dist: jieba; extra == "multilingual"
Requires-Dist: pyvi; extra == "multilingual"
Provides-Extra: litellm
Provides-Extra: tgi
Provides-Extra: optimum
Provides-Extra: quantization
Provides-Extra: adapters
Provides-Extra: nanotron
Provides-Extra: tensorboardx
Provides-Extra: vllm
Provides-Extra: quality
Provides-Extra: tests
Provides-Extra: dev
Provides-Extra: docs
Provides-Extra: extended-tasks
Provides-Extra: s3
Provides-Extra: multilingual
Description-Content-Type: text/markdown
License-File: LICENSE
[Description omitted; length: 5678 characters]

WHEEL

Wheel-Version: 1.0
Generator: setuptools (75.6.0)
Root-Is-Purelib: true
Tag: py3-none-any

RECORD

Path Digest Size
lighteval/__init__.py sha256=5NEYneDuYWDp83SJx108iOr1b-T08TkewQevIIeJK4E 1206
lighteval/__main__.py sha256=8Btr88y2FZhdGJjkESohPmBtqR7gkS5LPtyyz2j4WzQ 2859
lighteval/data.py sha256=fhfsdMEyg6i5_6MIwzn6oHh_AZ6uLBKbD4SwgalkDH0 13565
lighteval/main_accelerate.py sha256=V9IOm9UdtraFunvjX5G8LRwKS3eJvHPSxmS-xg7bl98 8964
lighteval/main_baseline.py sha256=FGvv4249Ra3_YxAANWyOCxpGlB-Jp4QcLuwMNCyamik 4916
lighteval/main_endpoint.py sha256=3-pn4cxJJqqAjnm-XWCtYhZcy5THBj1qI0u5wQwPBXw 18792
lighteval/main_nanotron.py sha256=DkIWlyBILSichvob9dFZg_k7JtUmyu6i9BAPIA5dsaQ 4781
lighteval/main_tasks.py sha256=IcP1WpKXVx-RO59ApU_NPdsS-D7GoQ1bsEXEAgFaq88 2988
lighteval/main_vllm.py sha256=ZqnBTHxEMsSaBYq-7Q42CptzmGrsWCLJV6ftYWO0q9A 6032
lighteval/pipeline.py sha256=QxXzdD6qmBjnEWdXK1gyCNTBtCUX78WRaPq--tUp9-I 15167
lighteval/config/__init__.py sha256=47DEQpj8HBSa-_TImW-5JCeuQeRkm5NMpJWZG3hSuFU 0
lighteval/config/lighteval_config.py sha256=oxa1lmQEv-4AIjpeM3Ljt5tZWLwxG3Cix5B1HVZPK48 3357
lighteval/logging/evaluation_tracker.py sha256=rL0hergDmSaToV1vCYBVq6n4Br17ZWNe16pZjUyGIAU 28953
lighteval/logging/info_loggers.py sha256=XCaog6s34j4FnJ7iWUFs5VgAn_bManhZjUMdF7_a9wg 29025
lighteval/metrics/__init__.py sha256=XcqtKnBGQeYp4V_pPjPVIUsZekDLkn2IxOhHliv7o8k 9423
lighteval/metrics/dynamic_metrics.py sha256=oJ0NHXpA1mq6NFAg-37Td8AHpFeDssxaqCmAEaY00JE 6376
lighteval/metrics/judge_prompts.jsonl sha256=PfMpr2gGvHRFp66RepgZ0YuiXnPBR22P7yjeLA__uv4 10451
lighteval/metrics/llm_as_judge.py sha256=MT6cIZmLJox1G3rot2YL7ngt2szTdYWk1_Xs6bGbH1k 10433
lighteval/metrics/metrics.py sha256=GiqXGt1y3WQJpq6ZWYKyPijLVxKllgMuqfX8ththUyg 22591
lighteval/metrics/metrics_corpus.py sha256=5TRLm66tafHRfWfW5TrVYbX2wmskiD86rW_3jad1wbs 6457
lighteval/metrics/metrics_sample.py sha256=qgpE3mZkiWo1jxz83Bmlp0YmfbUD9oqQCQhC6PEW8OA 44400
lighteval/metrics/normalizations.py sha256=zRqbQ1U5j9TlfynvYe6HyTQyvNnAsoJBPD6ObPa8lwg 16691
lighteval/metrics/sample_preparator.py sha256=cHzpxy8SZXjetiXuNDSX42vwoPsvQd9qGUp3Bjg1Eb8 5752
lighteval/metrics/stderr.py sha256=2Y51BYaaIR4zLwGeQLH5wTjKCi4M0c66bXAwWP3r320 4411
lighteval/metrics/harness_compatibility/drop.py sha256=JVQJewQFHjqsFVRPVYxdeCUzWpYHUGJmTicN4YI9YIM 7172
lighteval/metrics/harness_compatibility/truthful_qa.py sha256=7PfI5-LMEVdXOjvJqkgnV3FeHloMmlECYjR60m-BAOU 2832
lighteval/metrics/imports/bert_scorer.py sha256=XzV2iPTR3gi6Lb6FDondF4eVfbaL-DcUwAJQpocQUPw 18493
lighteval/metrics/imports/data_stats_metric.py sha256=QmZF5_FP6F_OtsPzHKavc63oTJw3LYz02xRfHF85MYo 5717
lighteval/metrics/imports/data_stats_utils.py sha256=eca-eXrrcyjZwcJbW0bmSYAY3jt-n827gKbIsTJnW58 6116
lighteval/metrics/imports/summac.py sha256=cW3YpHUt-g6qHxJz7ouU2WH9UFblJygXnzYp0jnkkb8 11683
lighteval/metrics/utils/linguistic_tokenizers.py sha256=8rqgyKEZuGxct85rY90AeXD6d5AeKOZ0no9ywcCimKI 11283
lighteval/metrics/utils/metric_utils.py sha256=jmWjzVsySC7mEvAeHbN_z6e2SnsPaEZD98FN0IpMWWE 3471
lighteval/models/__init__.py sha256=47DEQpj8HBSa-_TImW-5JCeuQeRkm5NMpJWZG3hSuFU 0
lighteval/models/abstract_model.py sha256=El4vYOh2qQ-zb7GqXwKJmwJBizVMgM9stklaf9L_mw4 8435
lighteval/models/litellm_model.py sha256=EUMzbkdN06YZOybNRv_kUm8Qv6fNVr2dswuzshiFA-A 11808
lighteval/models/model_input.py sha256=gcYN4rJTmQV9Y-vlBL8j9Sg3be7em_ixTCcaGTxA19k 5567
lighteval/models/model_loader.py sha256=om9Ypklm2KxX1E4O28NgZU_1iOYlG8lEhjFOOWa7zGs 6252
lighteval/models/model_output.py sha256=m35D0HVKPOtltOnJINZLxxX-WmIJJMjreqBA4khYIto 2915
lighteval/models/utils.py sha256=rmQ-KTXTDJuNrFsihvZifsjDdAkooc0v4JtKdDoB79A 3824
lighteval/models/dummy/dummy_model.py sha256=Khl5fhrpmz9j4m7tTMCiOLgfMMmbg0ItcMqpT6ou1qY 3602
lighteval/models/endpoints/endpoint_model.py sha256=aW0Uy6ThSgZn_uEvQ5IPe2v3zc_a1W8v20XaDjBxYvo 28511
lighteval/models/endpoints/openai_model.py sha256=QjCDjINC3DWhkq-_AfYdF_tsoAU2ymP6UWOVhlxOgJg 11155
lighteval/models/endpoints/tgi_model.py sha256=MOlkMKtnd2gcUryMm9CQACw01MTUDNJiq89mFBHvMrE 5878
lighteval/models/nanotron/nanotron_model.py sha256=SS8s0rObns90EgniKpfg1mjOqlpNpd1aOIEUzUVq1dQ 65089
lighteval/models/transformers/adapter_model.py sha256=gCHzKMK0b2WAyv1OKpJg51l-uthakRKh2HuYFBLP13U 5240
lighteval/models/transformers/delta_model.py sha256=fTunYtW-y_VXx5E4iJ2p__m-vBOay0Az53s3E6UASwU 4172
lighteval/models/transformers/transformers_model.py sha256=USOKI8WpC78_nYg9G-K8C9iAdzs5SiPLpJEmrG9mafo 65725
lighteval/models/vllm/vllm_model.py sha256=aj2-63GTi_gPGSQ7L_9E13GpZ6dB0RjPoHlFKFqJMcA 18279
lighteval/tasks/__init__.py sha256=47DEQpj8HBSa-_TImW-5JCeuQeRkm5NMpJWZG3hSuFU 0
lighteval/tasks/default_prompts.py sha256=HT4nacfwSEZk5vmriRB2lz_3ja3LtbauSvlyKCndg8A 103017
lighteval/tasks/default_tasks.py sha256=7u4rO9FZ9DF035w9nqzpd-T9S7NtSjc8sfK6pmrSXXg 666821
lighteval/tasks/lighteval_task.py sha256=-eDfSUNRP_CtrFzyIR8dO4kBmpP_HxzIYUd8IiQ2Zd4 28049
lighteval/tasks/prompt_manager.py sha256=81RYtOJUIgoUR0adb9iRN5dku_69jQ1d6tSeN5IPiTw 18752
lighteval/tasks/registry.py sha256=jSgU0bL246ZdKejQlCtIRkNk301jaUg5YG7HMlrQ1Uc 15226
lighteval/tasks/requests.py sha256=fdeO-vAuawrgLK0Gnos17YEjQhpwo5kXhwSEl2N0cz8 7096
lighteval/tasks/extended/__init__.py sha256=xdXHV5K28PNypPoeLojvezzT_vBNBZMzk-5KYCUSreY 1597
lighteval/tasks/extended/ifeval/instructions.py sha256=jZgpr8Jg6gx27IS5GF6aovVLQ-Wi1zFhinuN-tF8gUA 56053
lighteval/tasks/extended/ifeval/instructions_registry.py sha256=iRVMuVdoCBc3HyTHDxPkxi9hKOyZhWkEuxWSpa1Z4eM 7258
lighteval/tasks/extended/ifeval/instructions_utils.py sha256=08GaBGzAFUuwipF1fO70nORGh5799vtH4TdDHJdqvu8 25894
lighteval/tasks/extended/ifeval/main.py sha256=xNspHITl1GNxiPpl2oHYVzkzhFbuyirim0KQzSIMWds 5692
lighteval/tasks/extended/mix_eval/judge_prompts.py sha256=n5EciOYpKIJDz-mvb0LlT_qCF8O_ukXZiXeoo6IwEVA 13347
lighteval/tasks/extended/mix_eval/main.py sha256=WC2DMKyXWN3ml8DKxkHkubrqp0LQ0vh-Z7n3u_58_Kw 8015
lighteval/tasks/extended/mix_eval/prompts.py sha256=UNw1V2VNeUEFumnXdrh9SKn8W8u8zdoiZKjg_4wC6Ms 9704
lighteval/tasks/extended/mt_bench/judge_prompt_templates.py sha256=ytWhGfSma_ukwwBhEqDToljUWvPOHQXYPZAsuy4DLS8 6308
lighteval/tasks/extended/mt_bench/main.py sha256=tKdvtKP5mPZ_kaORiihgOvMFv5MpdPz1migaFOfqeJ0 3553
lighteval/tasks/extended/tiny_benchmarks/main.py sha256=6WJ3-ICFiDe1hv3tFYpMw_N4nCJPmG2xr0LTdGol8Bc 11176
lighteval/tasks/multilingual/adapters.py sha256=SUSouX4v1iw6pev1X_KIXDZoLX05YT8wx1HlY47GML4 12244
lighteval/tasks/multilingual/tasks.py sha256=x9A0argJPUtd2LoCNb8_-_3UOQfF63m2dSNgm4KoN_g 127582
lighteval/tasks/multilingual/utils/adapters_utils.py sha256=U9uiDViC9fhEHhGkTFlaRRGEV5KMDE8VHXmchiLC2GY 5476
lighteval/tasks/multilingual/utils/task_utils.py sha256=1y42LnO37VtEViYCWE6xLZA5OGW_I2mivsKjAJ1SCPI 1863
lighteval/tasks/templates/boolq.py sha256=f0_GJRdLqB6KxaRvBD3jv3FHZtP3pt1sLLe_AinQneg 4380
lighteval/tasks/templates/continuation.py sha256=jB-PSnI0g-4SND8T9fsq0ZGdAojO6DFOdlwkSQ0hpdk 8027
lighteval/tasks/templates/copa.py sha256=bkNl-f2LbKuUz6rgGnV8A_ZnrXOFyzr36r9Qo04n8l0 5983
lighteval/tasks/templates/hellaswag.py sha256=s-WjBgziZcswAbw9ivoFHApVYINzPUHUKjUpUN4B21k 6037
lighteval/tasks/templates/multichoice.py sha256=ohbv0-rXOFhiyFpPYSH_RVCr79BqubYP4RO_tB5J4Ug 6270
lighteval/tasks/templates/nli.py sha256=Q0aleY5rBxWurrwdDOlDw6o9M3U4v9CpzAmoAdDpnLg 11102
lighteval/tasks/templates/qa.py sha256=JcACl8lAATazRE-PRt1HH6gICRVxeyT89eTkc5iyamI 2801
lighteval/tasks/templates/translation.py sha256=l8M94xtEVZTMrIHlVmgU4MV1YlXfKZOn2a6StbqZ7B8 6034
lighteval/tasks/templates/utils/adapter_utils.py sha256=zoiTe5Z7KzJUQfU17702deVJjig32au49IWmb15_if8 1760
lighteval/tasks/templates/utils/formatting_utils.py sha256=5A8O9bD7Rt7eHZSLzS1nD8F8a2obh8S3D-Y5AwRx3Yw 4150
lighteval/tasks/templates/utils/formulation.py sha256=BA3ji-xhfhUUGcOAb5Ndw9flRJqrvrucLlhCxqfbw_E 5715
lighteval/tasks/templates/utils/translation_literals.py sha256=InI44ijmfRDpOR0w29j2_ha1XxI5miGBX-WvK0I82pg 32302
lighteval/utils/imports.py sha256=K66hI4JBG_3BPmGV9g_qY64DSVS3-lgbzO8S9KASrl8 4766
lighteval/utils/language.py sha256=OVIiXNfSp_Xb6N32ZjFnhNSy3MYN04hIHDURCTZci0g 6881
lighteval/utils/parallelism.py sha256=BK3xUbXhtOGjPe0RRvZiraU3DnKMqQwTWKdPz48lHwg 6273
lighteval/utils/utils.py sha256=vhYCLOBbgy6tYn9QHambueSV7OlyMH0ipgEX3ZzE0xw 7807
lighteval-0.7.0.dist-info/LICENSE sha256=tPk5wOXOUTsipCsjMpS1bCJqJ5Nudwj1HtE3xgLo6w8 1069
lighteval-0.7.0.dist-info/METADATA sha256=q5kbfq7HWzwlIA6G3BC-Fz2KUfVBw2U6jKbSeAnFPHw 8944
lighteval-0.7.0.dist-info/WHEEL sha256=PZUExdf71Ui_so67QXpySuHtCi3-J3wvF4ORK6k_S8U 91
lighteval-0.7.0.dist-info/entry_points.txt sha256=eW_fUc4_bJVN6LJSjI1wB39LbT6pllEOUEUbY3qPXlc 53
lighteval-0.7.0.dist-info/top_level.txt sha256=dvkN9L0K1BpVFmb-YCakjCmC13DtEVvsyMi5V12jdvg 10
lighteval-0.7.0.dist-info/RECORD — —

top_level.txt

lighteval

entry_points.txt

[console_scripts]
lighteval = lighteval.__main__:app