Wheelodex » Projects » lighteval

lighteval

View on PyPI — Reverse Dependencies (1)

0.6.2 lighteval-0.6.2-py3-none-any.whl

Wheel Details

Project: lighteval
Version: 0.6.2
Filename: lighteval-0.6.2-py3-none-any.whl
Download: [link]
Size: 335736
MD5: cac0fcf853048c13b7eef068bc0f3f43
SHA256: 1832fff4ca76d4ec617b5242c60e5dcaa1df8966f9b8352af105386fb6c910ba
Uploaded: 2024-10-23 14:11:46 +0000

dist-info

METADATA · WHEEL · RECORD · top_level.txt · entry_points.txt

METADATA

Metadata-Version: 2.1
Name: lighteval
Version: 0.6.2
Summary: A lightweight and configurable evaluation package
Author-Email: Clémentine Fourrier <clementine[at]huggingface.com>, Nathan Habib <nathan.habib[at]huggingface.com>, Thomas Wolf <thom[at]huggingface.com>
Maintainer-Email: Clémentine Fourrier <clementine[at]huggingface.com>, Nathan Habib <nathan.habib[at]huggingface.com>
Project-Url: Homepage, https://github.com/huggingface/lighteval
Project-Url: Issues, https://github.com/huggingface/lighteval/issues
License: MIT License
Keywords: evaluation,nlp,llm
Classifier: Development Status :: 3 - Alpha
Classifier: Intended Audience :: Developers
Classifier: Programming Language :: Python :: 3.10
Classifier: License :: OSI Approved :: MIT License
Classifier: Operating System :: OS Independent
Requires-Python: >=3.10
Requires-Dist: transformers (>=4.38.0)
Requires-Dist: huggingface-hub (>=0.23.0)
Requires-Dist: torch (<2.5,>=2.0)
Requires-Dist: GitPython (>=3.1.41)
Requires-Dist: datasets (>=2.14.0)
Requires-Dist: termcolor (==2.3.0)
Requires-Dist: pytablewriter
Requires-Dist: colorama
Requires-Dist: aenum (==3.1.15)
Requires-Dist: nltk (==3.9.1)
Requires-Dist: scikit-learn
Requires-Dist: spacy (==3.7.2)
Requires-Dist: sacrebleu
Requires-Dist: rouge-score (==0.1.2)
Requires-Dist: sentencepiece (>=0.1.99)
Requires-Dist: protobuf (==3.20.*)
Requires-Dist: pycountry
Requires-Dist: fsspec (>=2023.12.2)
Requires-Dist: accelerate; extra == "accelerate"
Requires-Dist: peft (==0.3.0); extra == "adapters"
Requires-Dist: lighteval[accelerate,multilingual,quality,tests]; extra == "dev"
Requires-Dist: langdetect; extra == "extended-tasks"
Requires-Dist: openai; extra == "extended-tasks"
Requires-Dist: tiktoken; extra == "extended-tasks"
Requires-Dist: stanza; extra == "multilingual"
Requires-Dist: spacy[ja,ko,th]; extra == "multilingual"
Requires-Dist: jieba; extra == "multilingual"
Requires-Dist: pyvi; extra == "multilingual"
Requires-Dist: nanotron; extra == "nanotron"
Requires-Dist: tensorboardX; extra == "nanotron"
Requires-Dist: optimum (==1.12.0); extra == "optimum"
Requires-Dist: ruff (==v0.2.2); extra == "quality"
Requires-Dist: pre-commit; extra == "quality"
Requires-Dist: bitsandbytes (>=0.41.0); extra == "quantization"
Requires-Dist: auto-gptq (>=0.4.2); extra == "quantization"
Requires-Dist: s3fs; extra == "s3"
Requires-Dist: tensorboardX; extra == "tensorboardx"
Requires-Dist: pytest (==7.4.0); extra == "tests"
Requires-Dist: text-generation (==0.6.0); extra == "tgi"
Requires-Dist: vllm; extra == "vllm"
Requires-Dist: ray; extra == "vllm"
Requires-Dist: more-itertools; extra == "vllm"
Provides-Extra: accelerate
Provides-Extra: adapters
Provides-Extra: dev
Provides-Extra: extended_tasks
Provides-Extra: multilingual
Provides-Extra: nanotron
Provides-Extra: optimum
Provides-Extra: quality
Provides-Extra: quantization
Provides-Extra: s3
Provides-Extra: tensorboardx
Provides-Extra: tests
Provides-Extra: tgi
Provides-Extra: vllm
Description-Content-Type: text/markdown
License-File: LICENSE
[Description omitted; length: 5048 characters]

WHEEL

Wheel-Version: 1.0
Generator: setuptools (75.2.0)
Root-Is-Purelib: true
Tag: py3-none-any

RECORD

Path Digest Size
lighteval/__init__.py sha256=5NEYneDuYWDp83SJx108iOr1b-T08TkewQevIIeJK4E 1206
lighteval/__main__.py sha256=BzVaHU7eovNbv59LU3iyYv_0vwEC_s7fjlifH9N-Lq4 4139
lighteval/data.py sha256=9MhKYYjmsfPZ-pml0McejRz9iHZs_UoxTmg6rxFSNqI 13290
lighteval/main_accelerate.py sha256=0OvzvCmIuIloDJ987ooWNdiUqhIY44jrxPPR3kjnBNQ 3435
lighteval/main_baseline.py sha256=cX3fOqU601vKPGaYfMUa0UCZoZXdM49nbYTCEgBA5Sc 3735
lighteval/main_nanotron.py sha256=ErVCsVOTazEhSAr0R7W1vmdySF-wxwF58vsuAJSzT14 4226
lighteval/parsers.py sha256=S-LpO034YXL9OooI3-l0zxT91OVKuWIk8I_HpMjUzsA 7396
lighteval/pipeline.py sha256=siGi182Fn0P_dkNoExGkklAFsHx6Xi7RLPkwTVegDfQ 14979
lighteval/config/__init__.py sha256=47DEQpj8HBSa-_TImW-5JCeuQeRkm5NMpJWZG3hSuFU 0
lighteval/config/lighteval_config.py sha256=9JNIaW0g1Z6dU64fABZxUlKn0LAt-VSMfG5yqTJTRds 3244
lighteval/logging/evaluation_tracker.py sha256=Yzu6Cmz3SEWD20GdOrXdv6deNERbFyBRCmJxRfohOwc 28168
lighteval/logging/hierarchical_logger.py sha256=4MofgOCYypt5GDWfMEwIdBfw3aM8dAm5pfUp1Zu3Vng 5679
lighteval/logging/info_loggers.py sha256=HVZi_fkN8Vm3vdNd4iDIjrLRllxdS105c6_nQ8kAl4c 29021
lighteval/metrics/__init__.py sha256=ej7aJbNneQRcI-mCQFblCDirSDl_T2DbEJ3Yg4DWas0 9600
lighteval/metrics/dynamic_metrics.py sha256=oJ0NHXpA1mq6NFAg-37Td8AHpFeDssxaqCmAEaY00JE 6376
lighteval/metrics/judge_prompts.jsonl sha256=PfMpr2gGvHRFp66RepgZ0YuiXnPBR22P7yjeLA__uv4 10451
lighteval/metrics/llm_as_judge.py sha256=xzFbEtzCBr-wUMLd8byW7ikAc5HFn0dBrMELuAtBlDI 9162
lighteval/metrics/metrics.py sha256=GiqXGt1y3WQJpq6ZWYKyPijLVxKllgMuqfX8ththUyg 22591
lighteval/metrics/metrics_corpus.py sha256=uhIJ62L-g0dwKyCnQIepa42hHArF9bjnOywF_4WWh7s 6125
lighteval/metrics/metrics_sample.py sha256=NTtNorHzxX4w36_vHW26_tZLVKFZ4KVWL5c3PQIL2HE 44138
lighteval/metrics/normalizations.py sha256=zRqbQ1U5j9TlfynvYe6HyTQyvNnAsoJBPD6ObPa8lwg 16691
lighteval/metrics/sample_preparator.py sha256=VZIZKB9b06DHG6w7-vGqeNPu6OdPA8mULY5XiRLzVbI 5736
lighteval/metrics/stderr.py sha256=dxR6mjQhuuuS2lHGPpCFjc583OLiQqR6FLBprddon0c 4387
lighteval/metrics/harness_compatibility/drop.py sha256=JVQJewQFHjqsFVRPVYxdeCUzWpYHUGJmTicN4YI9YIM 7172
lighteval/metrics/harness_compatibility/truthful_qa.py sha256=7PfI5-LMEVdXOjvJqkgnV3FeHloMmlECYjR60m-BAOU 2832
lighteval/metrics/imports/bert_scorer.py sha256=AU12SlHS9vcjgVrXJTnXRQff1FOk3Tx9vMn7ilNwYCE 18484
lighteval/metrics/imports/data_stats_metric.py sha256=1T3qzV0s9Oomz5b58I-1IO20OoFDbgsldvGX_FS6DB8 5714
lighteval/metrics/imports/data_stats_utils.py sha256=eca-eXrrcyjZwcJbW0bmSYAY3jt-n827gKbIsTJnW58 6116
lighteval/metrics/imports/summac.py sha256=3JLdMvw1yrMce6OTwL-J5NAvyvzxh4-ri2FEpeJrl48 11679
lighteval/metrics/utils/linguistic_tokenizers.py sha256=3FTziFycNX4DmTJQhZgASyIGfS88FniKX-J3LzM99jY 11284
lighteval/metrics/utils/metric_utils.py sha256=jmWjzVsySC7mEvAeHbN_z6e2SnsPaEZD98FN0IpMWWE 3471
lighteval/models/__init__.py sha256=47DEQpj8HBSa-_TImW-5JCeuQeRkm5NMpJWZG3hSuFU 0
lighteval/models/abstract_model.py sha256=El4vYOh2qQ-zb7GqXwKJmwJBizVMgM9stklaf9L_mw4 8435
lighteval/models/adapter_model.py sha256=XInSyilGyqj2ZwNoT3Ul3hhAiVn5rXp8mFgRUIUC4Cs 3567
lighteval/models/base_model.py sha256=hXpbYNwh2CWMMHgU27leeHHkKIAQ_R9GO3ttgEYk80w 51836
lighteval/models/delta_model.py sha256=S6xCq7Ym19XZgJOkOPwwqPPZUp-RTzfu1sswGZU1Npo 3492
lighteval/models/dummy_model.py sha256=raMkPOruSUqMhY0tVzTwoGJ8jG5RtuadIc0ldOlBx8s 3571
lighteval/models/endpoint_model.py sha256=snHx-hNvTxrQJbj7ub4wwoldVXxhujWIIGFbb2OITGo 17017
lighteval/models/model_config.py sha256=A0ogWUoqmiFGqg67fMwGUBPyul5PcXxJmsQMHyvvqlk 20478
lighteval/models/model_loader.py sha256=r_sgD8bmatPZvxbM5PIISB-9UDhW8hofGLHBSypRBVM 5633
lighteval/models/model_output.py sha256=m35D0HVKPOtltOnJINZLxxX-WmIJJMjreqBA4khYIto 2915
lighteval/models/nanotron_model.py sha256=gLwZSU3ciqYNCZKzWiJjuySbyCrsSC7m52iCqncf7bE 65070
lighteval/models/openai_model.py sha256=bgrd-g8KfS36RCjqZF_uzoqa06_XNa7gSL_FLQw7ye8 10352
lighteval/models/tgi_model.py sha256=mFV5QT0PHmIZXP7UqPortOLDm4lUrtUqjewn3ozrj7s 4369
lighteval/models/utils.py sha256=rmQ-KTXTDJuNrFsihvZifsjDdAkooc0v4JtKdDoB79A 3824
lighteval/models/vllm_model.py sha256=wHMwS2g5XHq6_3waSCxvMIufRVe4RnLNR180RhmU1Ow 16488
lighteval/tasks/__init__.py sha256=47DEQpj8HBSa-_TImW-5JCeuQeRkm5NMpJWZG3hSuFU 0
lighteval/tasks/default_prompts.py sha256=QohsExjABbdnD-JPtrJs4TtOwv1QOrEJ35trv8nXrr4 103419
lighteval/tasks/default_tasks.py sha256=diG2TVuxMC5AkcdEvKkVXXMiQAP74kTbKQcNYDLbG7k 715907
lighteval/tasks/lighteval_task.py sha256=mCLYcemzdHPnNpPZAPsfJ5EZPmmsQC0DgcXoRio9Rck 28548
lighteval/tasks/prompt_manager.py sha256=e8yqHBVwNjjKiJKu2pKqB0A9Q6SmXxB5By2UvxR2CNk 17591
lighteval/tasks/registry.py sha256=BLXmzHjZpI4YyQQhgFR4oEMcTolCKCnJC798LjI1UFY 15273
lighteval/tasks/requests.py sha256=yYHhKCbt_V1n8cuQbqbyZYNWf1xZ09_HyBo3475P_h8 7400
lighteval/tasks/extended/__init__.py sha256=xdXHV5K28PNypPoeLojvezzT_vBNBZMzk-5KYCUSreY 1597
lighteval/tasks/extended/ifeval/instructions.py sha256=jZgpr8Jg6gx27IS5GF6aovVLQ-Wi1zFhinuN-tF8gUA 56053
lighteval/tasks/extended/ifeval/instructions_registry.py sha256=iRVMuVdoCBc3HyTHDxPkxi9hKOyZhWkEuxWSpa1Z4eM 7258
lighteval/tasks/extended/ifeval/instructions_utils.py sha256=08GaBGzAFUuwipF1fO70nORGh5799vtH4TdDHJdqvu8 25894
lighteval/tasks/extended/ifeval/main.py sha256=pAn0zGQPVAlDXAkv32cIV-E7XGAFl_0KJoAEhnIaIGs 5832
lighteval/tasks/extended/mix_eval/judge_prompts.py sha256=n5EciOYpKIJDz-mvb0LlT_qCF8O_ukXZiXeoo6IwEVA 13347
lighteval/tasks/extended/mix_eval/main.py sha256=9Qv3p4j-K5JND8A_rwiQ39RNddSq1POuor2wLO_ht6U 8155
lighteval/tasks/extended/mix_eval/prompts.py sha256=UNw1V2VNeUEFumnXdrh9SKn8W8u8zdoiZKjg_4wC6Ms 9704
lighteval/tasks/extended/mt_bench/judge_prompt_templates.py sha256=ytWhGfSma_ukwwBhEqDToljUWvPOHQXYPZAsuy4DLS8 6308
lighteval/tasks/extended/mt_bench/main.py sha256=29wtnTlvf2ChsjKKAKDBIW4XLmksHEhLJd1Nhdd_tXg 3651
lighteval/tasks/extended/tiny_benchmarks/main.py sha256=ioZQWAA6_xDrneaMnTShQUi5KqmZJGZwk_nh73DjXVw 11358
lighteval/tasks/multilingual/adapters.py sha256=SUSouX4v1iw6pev1X_KIXDZoLX05YT8wx1HlY47GML4 12244
lighteval/tasks/multilingual/tasks.py sha256=och58XtS4TulZ_GrGWAopQsVSQJWnebrXNkFMfI0Qbg 124815
lighteval/tasks/multilingual/utils/adapters_utils.py sha256=U9uiDViC9fhEHhGkTFlaRRGEV5KMDE8VHXmchiLC2GY 5476
lighteval/tasks/multilingual/utils/task_utils.py sha256=1y42LnO37VtEViYCWE6xLZA5OGW_I2mivsKjAJ1SCPI 1863
lighteval/tasks/templates/boolq.py sha256=f0_GJRdLqB6KxaRvBD3jv3FHZtP3pt1sLLe_AinQneg 4380
lighteval/tasks/templates/continuation.py sha256=UJJBkbDZR3fmwkOjtDewRJY0kAOumJGUTWqH4PT7TQw 7643
lighteval/tasks/templates/copa.py sha256=bkNl-f2LbKuUz6rgGnV8A_ZnrXOFyzr36r9Qo04n8l0 5983
lighteval/tasks/templates/hellaswag.py sha256=s-WjBgziZcswAbw9ivoFHApVYINzPUHUKjUpUN4B21k 6037
lighteval/tasks/templates/multichoice.py sha256=ohbv0-rXOFhiyFpPYSH_RVCr79BqubYP4RO_tB5J4Ug 6270
lighteval/tasks/templates/nli.py sha256=Q0aleY5rBxWurrwdDOlDw6o9M3U4v9CpzAmoAdDpnLg 11102
lighteval/tasks/templates/qa.py sha256=JcACl8lAATazRE-PRt1HH6gICRVxeyT89eTkc5iyamI 2801
lighteval/tasks/templates/utils/adapter_utils.py sha256=zoiTe5Z7KzJUQfU17702deVJjig32au49IWmb15_if8 1760
lighteval/tasks/templates/utils/formatting_utils.py sha256=5A8O9bD7Rt7eHZSLzS1nD8F8a2obh8S3D-Y5AwRx3Yw 4150
lighteval/tasks/templates/utils/formulation.py sha256=BA3ji-xhfhUUGcOAb5Ndw9flRJqrvrucLlhCxqfbw_E 5715
lighteval/tasks/templates/utils/translation_literals.py sha256=o-uSZ-Mvygr_kVL-OvoawZzbxX19vHYykv0EZxLzRxM 28875
lighteval/utils/imports.py sha256=kv0SjjVz_I1PnAGQzok2hgFJyfUohHmPPg2eFS_58Z8 4500
lighteval/utils/language.py sha256=cH6t3SNSn14fmXHsuXacpONrZmNGAwec6WXNk0bFhwU 6803
lighteval/utils/parallelism.py sha256=i3RkzgN2Trqpx8OGevfacMS953ynz2GMWqxqMD-tVeE 6254
lighteval/utils/utils.py sha256=ydlqnTH1PS33Zj1XDTlXzi0yUfcbkGfHuaj_i6xNBE4 7802
lighteval-0.6.2.dist-info/LICENSE sha256=tPk5wOXOUTsipCsjMpS1bCJqJ5Nudwj1HtE3xgLo6w8 1069
lighteval-0.6.2.dist-info/METADATA sha256=O-FoyhPglXY8MUWBGZ2_tKbfDJgBndzxqjpRiGgLXqc 8111
lighteval-0.6.2.dist-info/WHEEL sha256=OVMc5UfuAQiSplgO0_WdW7vXVGAt9Hdd6qtN4HotdyA 91
lighteval-0.6.2.dist-info/entry_points.txt sha256=1N4ZfVgpCcrBn9KEDWkTuMdh-2cnSOwEObt2b2YLcio 62
lighteval-0.6.2.dist-info/top_level.txt sha256=dvkN9L0K1BpVFmb-YCakjCmC13DtEVvsyMi5V12jdvg 10
lighteval-0.6.2.dist-info/RECORD — —

top_level.txt

lighteval

entry_points.txt

[console_scripts]
lighteval = lighteval.__main__:cli_evaluate