Wheelodex » Projects » llmebench » llmebench-1.0.1-py3-none-any.whl

llmebench

View on PyPI — Reverse Dependencies (0)

1.0.1 llmebench-1.0.1-py3-none-any.whl

Wheel Details

Project: llmebench
Version: 1.0.1
Filename: llmebench-1.0.1-py3-none-any.whl
Download: [link]
Size: 133020
MD5: 75e1274ac352bddab8c75dbeaba292b9
SHA256: 3832fec70016ecad9c9e7c5e6c689db8122e9438390245863a4e6af8bb547e38
Uploaded: 2024-08-12 17:47:20 +0000

dist-info

METADATA · WHEEL · RECORD · top_level.txt

METADATA

Metadata-Version: 2.1
Name: llmebench
Version: 1.0.1
Summary: A Flexible Framework for Accelerating LLMs Benchmarking
Author: Fahim Dalvi
Author-Email: faimaduddin[at]hbku.edu.qa
Home-Page: https://llmebench.qcri.org
Project-Url: Documentation, https://github.com/qcri/LLMeBench
Project-Url: Bug Tracker, https://github.com/qcri/LLMeBench
Classifier: Programming Language :: Python :: 3
Classifier: License :: OSI Approved :: BSD License
Classifier: Operating System :: OS Independent
Requires-Python: >=3.8
Requires-Dist: datasets (==2.14.6)
Requires-Dist: nltk (==3.8.1)
Requires-Dist: openai (==1.35.10)
Requires-Dist: anthropic (==0.31.2)
Requires-Dist: pandas (==2.0.2)
Requires-Dist: pooch (==1.7.0)
Requires-Dist: python-dotenv (==1.0.0)
Requires-Dist: scikit-learn (==1.2.2)
Requires-Dist: tenacity (==8.2.2)
Requires-Dist: websockets (==11.0.3)
Requires-Dist: evaluate (==0.4.2)
Requires-Dist: rouge-score (==0.1.2)
Requires-Dist: absl-py (==2.1.0)
Requires-Dist: GitPython (==3.1.43)
Requires-Dist: numpy (<2)
Requires-Dist: langcodes (==3.3.0); extra == "dev"
Requires-Dist: pytest (==7.3.1); extra == "dev"
Requires-Dist: pytest-cov (==4.1.0); extra == "dev"
Requires-Dist: pytest-subtests (==0.11.0); extra == "dev"
Requires-Dist: ufmt (==1.3.2); extra == "dev"
Requires-Dist: langchain (==0.0.198); extra == "fewshot"
Requires-Dist: sentence-transformers (==2.2.2); extra == "fewshot"
Requires-Dist: faiss-cpu (==1.8.0); extra == "fewshot"
Provides-Extra: dev
Provides-Extra: fewshot
Description-Content-Type: text/markdown
[Description omitted; length: 10833 characters]

WHEEL

Wheel-Version: 1.0
Generator: setuptools (72.1.0)
Root-Is-Purelib: true
Tag: py3-none-any

RECORD

Path Digest Size
llmebench/__init__.py sha256=1XHoxmzdnxg8V9Exlbry95WcC6OM8QuSSG97qrNDA_U 33
llmebench/__main__.py sha256=q7B3jc_BMq0pJ_RNsxmsealu33a1_-vBYCS_SXTvvTk 36
llmebench/asset_utils.py sha256=U0-unyudUyRx7uF62VM5orYNVXKfjomlWj6xZKLr2BA 1963
llmebench/benchmark.py sha256=TthWPCEk5XwUSjTR6KPcvets5-OLMIPDW9Cf1e-_Wqs 19507
llmebench/utils.py sha256=rfA7h9h8_CHo7xHFP9Y3Y78gbtV7A0KIvp8KOrgGaXs 6843
llmebench/datasets/ADI.py sha256=kKAunapshmY-hsatNKRs_LMJyf7Pkvisi0sKuKg8rmY 1627
llmebench/datasets/ANERcorp.py sha256=leiKaEiIwujtn7Wzow-ov20EYGOkX9nL7G9yOrZ0iCY 5540
llmebench/datasets/ANSFactuality.py sha256=jnxzUbiDR3piE2dHlNl5xvMUH8t40mrn0aEyLyLpJF4 3324
llmebench/datasets/ANSStance.py sha256=_7j-WrYXRgd-YqUvSP-KJe1QrkeNXxTsl_ydwkZJdpY 3337
llmebench/datasets/ARCD.py sha256=4lGgqV30tOqNd1rUA4hZNG7BAo1vZxl1L8SykCqwb38 2422
llmebench/datasets/ASND.py sha256=AwyY1b-a2Eny8KbcXr_n9DgCFS2ZUnHV67CukiDzvQE 2544
llmebench/datasets/Adult.py sha256=6TK2bCqbeUtyyZXBSQe5JRY6nFCXRdNEbXdpJ8wmB6c 2110
llmebench/datasets/Aqmar.py sha256=QkdguSIgn9VgV499VN3J1uLxZ49Nux58ze--ksUsSrA 4793
llmebench/datasets/ArMemes.py sha256=xbSU0h1f_OP6eksAPIHVbA8TaHbB2ue9XGhYUyTX9u0 2074
llmebench/datasets/ArSAS.py sha256=wgV0wxcUpRrB73EsBgTBL5jo_MnTU9wW31rcasmvgyw 1431
llmebench/datasets/ArSarcasm.py sha256=e8EoZ_NKtYrM-hIy4ELoj8D2Azob5Rq6F-GriDCTXNE 2205
llmebench/datasets/ArSarcasm2.py sha256=JovL-oIBNy7WstppzpVW-xj49CSPqJzm42WslDXlktE 1898
llmebench/datasets/AraBench.py sha256=_tQziEF9wZAA-wEELxN2YXU3VLjg6FjtOpeHunsn7FM 12623
llmebench/datasets/ArabGend.py sha256=l6U0vYfL317BJ5PufK21oQHjVPkexVitALa0Ddu47GE 2796
llmebench/datasets/ArapTweet.py sha256=jiwVSHbYfN7gp69S-DKaPbjACWZhIryLzLO1yhcWbVA 3678
llmebench/datasets/BanFakeNews.py sha256=iJI7R8MyU0U57u8ZbZ3uinxKANOywp8pq-0qafy6jDM 1763
llmebench/datasets/BanglaNewsCategorization.py sha256=iDWDcCPbPq0y99bQTdSsacirZAYYHL3TTkrRiPv7d9w 2906
llmebench/datasets/BanglaSentiment.py sha256=1-t_qVosDW51y1kz0zIUfwM7sMPiEtSaySR5nFy4170 2194
llmebench/datasets/BanglaVITD.py sha256=AndDzUBkYn11UiIJcQiFgaucA8Q2wXQVNemdJ33s0oU 2612
llmebench/datasets/BibleMaghrebiDiacritization.py sha256=WJazD_a87cr7h1ajBd5CssyF6-fFWgs1e35cV0h327Y 2541
llmebench/datasets/COVID19Factuality.py sha256=WN00LCMzGxasydM3ex1Msk2c2_6RNHepO2NNhYXB2Ts 2757
llmebench/datasets/CSV.py sha256=HAajXycAnwHDiRcHjNtCsbbxKwbU0yToyi29dDNK1sc 3901
llmebench/datasets/CT22Attentionworthy.py sha256=E58RAqLRjvCxeyI2J-eG7gF-p6zFXy3zNTB8OlGZlWI 2657
llmebench/datasets/CT22Checkworthiness.py sha256=OmsEPJwTFGCLQJRPOQsR9RnyAVpWweFFXgp84D6ZE7U 3484
llmebench/datasets/CT22Claim.py sha256=g0vbGSsH6EVMuAW5zXrrpOMGpIIY8L8fBwatQ-SbiJg 2192
llmebench/datasets/CT22Harmful.py sha256=5clrIk_hAQ3zloOBbJuAWluW2oHb5_0rs0aqB7bYELk 2202
llmebench/datasets/CT23Subjectivity.py sha256=SW8jO0GB75tR1Z6qNxwo3b4fQ-Jyyy4FndAlzGntqnk 2057
llmebench/datasets/Emotion.py sha256=4wXE4qgrdO2Td9RXsi9PITN3vkIdKuhjm4_WVoZCAOA 3730
llmebench/datasets/HuggingFace.py sha256=vpr-XeFo0hZRTOczc0wTLMKKTSk1Tu21xFwSjA5_ops 3521
llmebench/datasets/JSONL.py sha256=MYNqAjcUrxeURJL217aVhxMmh2buvneiNcqZXFPGPy8 2285
llmebench/datasets/Location.py sha256=aLTW4BZKIXyMm5vhhBO03dCI1iNDScqUiCQQgfGDeUg 2148
llmebench/datasets/MGBWords.py sha256=ThHOvrUs0rRlqUHlZd1QHD2mPX2PgWtoI30xCf_s_mA 4130
llmebench/datasets/MLQA.py sha256=9v2yPwXOO4z4ROyGmUF4yOk0bZtWxN1MJD7wv2pGbs4 2472
llmebench/datasets/MultiNativQA.py sha256=HOA52KoTg4KHNnZyDvy67r5o33kfYVdggd8Fu-ZHmGc 3428
llmebench/datasets/NameInfo.py sha256=PV--g0Num-1pfxNtBiNrUqTpOvblwui0djFIdtvy4EA 3506
llmebench/datasets/OSACT4SubtaskA.py sha256=Rgxr6E3qFQYX0p4hG_D_bTeb5tzk-BLujdcVbP24onM 3198
llmebench/datasets/OSACT4SubtaskB.py sha256=ozUErFUtWW1HlAeqvivpFElFPYNVx8dOGIMt380Ef-8 1962
llmebench/datasets/PADT.py sha256=FWJelMWvaV0uCjodzfsCU2OjFt9rlyNPfidD63PQrq4 3113
llmebench/datasets/QADI.py sha256=dkfh3fism6uJxgK69RTSfrfiTr7HZn20nfII_knsODY 2177
llmebench/datasets/QCRIDialectalArabicPOS.py sha256=j8i28HGGwB0oQsrYi_0AleA2W2kFIqx56xYEcXW1I-o 3723
llmebench/datasets/QCRIDialectalArabicSegmentation.py sha256=MbC-TqYP4kWJVIqrKKXD3rl-zwq33xOXCP2hfKQrbqo 2915
llmebench/datasets/SANADAkhbarona.py sha256=k97OZwOEMCOOTxsFXG_9XGhmoZmqRn4ZqqacJfRYcOg 2044
llmebench/datasets/SANADAlArabiya.py sha256=nEvUfOZ7OlXwbl5rvd_Sh8rEph2urka-sRD7nLwOAME 2016
llmebench/datasets/SANADAlKhaleej.py sha256=3vKVdazKwVGW5vuKnhGSkj2r39gipYaRzDw5Qyod7gw 2044
llmebench/datasets/SQuADBase.py sha256=VZqrmGPaNe9_i92k5_-rKIG3qvVRLyYolIv3U27Rai0 1549
llmebench/datasets/STSQ2Q.py sha256=bQMldJ4jMlClubPerjKsMntrcZLS_1uuSMy6_K7qyBw 1842
llmebench/datasets/SemEval17T1STS.py sha256=sLxVUTcK9iUoji9C_EqN5CyYxwfPx3ag1E8Gb6Bz01w 2721
llmebench/datasets/SemEval17T2STS.py sha256=Y-9eneCoFR7u1Imrwd-olOWG2TtD4UMGhNGXr1WB444 2718
llmebench/datasets/SemEval23T3Propaganda.py sha256=HVlxXnxFIoAurYXbJGlKzKQ8tjTw9HiV6pBuxkI-mXI 6615
llmebench/datasets/Spam.py sha256=YegfZ345kO3fpKKjdr2Nky8f5NCp_AqVbeP_e6maaOM 1638
llmebench/datasets/TSV.py sha256=zYCIusuHY4SuAbgME2ptdN3_YNGs6RHKUkcEuVSTxw8 1398
llmebench/datasets/ThatiAR.py sha256=hEFwKc19jq04YV4TQ3C_XVgd9jHhoIir2Yu2DTge_YY 1650
llmebench/datasets/TyDiQA.py sha256=-5UlUdnrDiDpwJA67r1MyHV04ef3OwzMQ9MZ2IcfMLU 1129
llmebench/datasets/UnifiedFCFactuality.py sha256=VTpAeZdDvuuKHxLEKrA_EVq6-gmuhnG2UzqfXGA6PqE 3073
llmebench/datasets/UnifiedFCStance.py sha256=UD3vjieJY2LKyoGQqu81NKz558N236Wq7WBZ20AYnAU 3439
llmebench/datasets/WANLP22T3Propaganda.py sha256=fU5GcfWQbhtQo75vXVcjXoad7mfT-a7uMWZwoUM2Qx8 4132
llmebench/datasets/WikiNewsDiacritization.py sha256=w82y9DeSR0dzdQEMg-zxBnYmtMBmc9OPdnrpNiy9J9s 2046
llmebench/datasets/WikiNewsLemmatization.py sha256=zOWJzAOkKglnop2qMyTjtmUtKdTvPjecUrVoDX3ZuiY 2114
llmebench/datasets/WikiNewsPOS.py sha256=pkL075-ttXd4lDce0rYXZGLzfMw5cWJmasD-U8MK-_Y 2492
llmebench/datasets/WikiNewsSegmentation.py sha256=BnInJ5wLunFXY1ruNQyxzfLdQS00ljMczD5fe4p_L2s 1715
llmebench/datasets/XGLUEPOS.py sha256=tdS6zks4ZO6QUWcq3cLTVLjBEgs2sHQoTXddOV3FnvE 2275
llmebench/datasets/XNLI.py sha256=qpT-8USRSTuDP5GN181WIYUJNZDOp5NpCdi-Ch-nDWY 2435
llmebench/datasets/XQuAD.py sha256=g1m9SbA3h5_ninAAX9rHvGAmdfgbTMNJlbJbBr4I1Qs 1044
llmebench/datasets/__init__.py sha256=9qPRrcD_J_fMiMGwAoashysmssf_M9dnoHkI0rxnvxc 2766
llmebench/datasets/dataset_base.py sha256=tVJO9er0mHnTZc5P7SRYwiZSGRU_lfsHK2Esi1x5ZT8 18136
llmebench/models/Anthropic.py sha256=p6JlgR3xGTtVjzuCvpcAO9BC0BxgCgiIC58_QOSnTjQ 3790
llmebench/models/AzureModel.py sha256=tLaGJuM6leL_sg0WscEFY_0QcGZ74qkJrenNN6KB0CE 4743
llmebench/models/FastChat.py sha256=CEKz67iCTfnOzQth3ei9BofYkbzQYjivwIjIPsoXjEQ 2039
llmebench/models/HuggingFaceInferenceAPI.py sha256=N-DKKG6rChNzJgsaCWVvv8vN36nraFdkaxzJY22CpUU 5548
llmebench/models/OpenAI.py sha256=b_WX6jyxOKc2WpwoAMJQg1bfGTYaj1JoMdUO173upeM 9017
llmebench/models/Petals.py sha256=udb5EBaGeTKxMWR5yEtxzsma9LJ5yD7ZcU0m8nFIm4E 4220
llmebench/models/Random.py sha256=EkSJAkMHMq4SSqptLec8GxHFUnHZyQZR0me6cIJd3sg 3020
llmebench/models/VLLM.py sha256=Ug50ba5GpVJ5n0lww_tVjcBBc4ZU-ToxjgTin2lDIxs 2714
llmebench/models/__init__.py sha256=CE2hr6GY-FfkKJLUjS7iXCErHzAi4JbhYJKuw4L1_Cs 340
llmebench/models/model_base.py sha256=SQk5H_jzQmSCxgrH2txGRkv7TQPtyqkEEFg9bmd1dpc 4459
llmebench/tasks/Adult.py sha256=aU9c22HnmwdBOBSTbnklacSmn0ZnG0PlCIQmJrIh5K0 486
llmebench/tasks/ArabicDiacritization.py sha256=kbxSqef84-RLCUY9qOegsGxxht0iXdSZMaeAMVT3m5g 4590
llmebench/tasks/ArabicPOS.py sha256=H4h5uXuzgErus3Idr0eHJq72uCKAn99_7vhQiZVXyP0 1148
llmebench/tasks/ArabicParsing.py sha256=McD3334OR4W9YqmOoWYaOgyzLc4BhrR8nhPyeJwVq90 844
llmebench/tasks/ArabicSegmentation.py sha256=ryEwPUHS40iatjtrcpThx1vA0I89t_xzLHN769G97bo 1015
llmebench/tasks/Attentionworthy.py sha256=kFRmYm-C2vyBrIVYtcKoxr83MffDgpzBn7zR4FTQhSA 1015
llmebench/tasks/Checkworthiness.py sha256=3rQ4dY37i_zWSvyIttTGJ2RIFNaShHT8gYK-BLWqNu4 1085
llmebench/tasks/ClaimDetection.py sha256=VaYNPXy-YosRA9ENzr_am8DS8VQN5arZaHDxosU9lTs 560
llmebench/tasks/Classification.py sha256=H-hYihtgOgCM8CzZgusKJTwOJiHZ6GlYfAmnAzKqgj0 1769
llmebench/tasks/DemographyGender.py sha256=trR38tPaxxk0ER8tR-H82C7rxvbtPoPBM24eqQRiZPY 508
llmebench/tasks/DemographyLocation.py sha256=I9XsQS7pa_R5bGgQXT4ThQGwbQL5oF1tv5Tm0XdnBxI 512
llmebench/tasks/DemographyNameInfo.py sha256=R488RmUUNE_leuq8IxKsdC3UarMjZCed7AtBP3aydsc 540
llmebench/tasks/DialectID.py sha256=aCtxs76evzhYFtN6HAFZVlK_OkZCXJDie4ENCXt_tlU 883
llmebench/tasks/Emotion.py sha256=WsUxBxZHUl3kcZ2PybE3GLjIWcwGupO7HtmmPPUAw10 536
llmebench/tasks/Factuality.py sha256=SGxH510ILAingAvyJu35VpVBxGdNsFlMF9RaW9FVvKs 1237
llmebench/tasks/HarmfulDetection.py sha256=9-v0rW3C6I3B9Kgke_0yXzSXK5vryI8NPFEQHkjI8no 583
llmebench/tasks/HateSpeech.py sha256=X7nOGnEF6UXRAKeZaeEomdF5U07yxHz1jeEjoixNueo 496
llmebench/tasks/Lemmatization.py sha256=IAldYGzfqQro8blZ0qvfKP_IW4jxb4V5v4fdbKi9qmA 739
llmebench/tasks/MachineTranslation.py sha256=uFyNWqlJ6mVaoJwurYP0VR4ZYjyORNoo2QjYLc8rg0k 453
llmebench/tasks/MultiNativQA.py sha256=i0f60tn6CFN9GzTRoyLzZoaIfw-JVZKzu_QXFbuXgys 1272
llmebench/tasks/MultilabelPropaganda.py sha256=aKjumIxFBNBLZFbNgSVOJ4MLbuOj8UbEZXjud3_kTvQ 1376
llmebench/tasks/NER.py sha256=A8i8EtUhwSmhxwnPJ-KcTGhDGL7X5T-YjivSPAIjg2o 1689
llmebench/tasks/NewsCategorization.py sha256=APvpRSW58E7bnbi8y0JFAAWvBmqE7VBjJ1WQeBhXqYg 1035
llmebench/tasks/Offensive.py sha256=IiUmZrqA9KhIDeyWc__NBTYKyWdclgdn4GlN5pVr9PM 494
llmebench/tasks/Q2QSimDetect.py sha256=FqbRFvQT1r3YA1h5ilXBaPER2sVqjXSAzMSDG_Xy2J4 507
llmebench/tasks/QA.py sha256=LEcryQkhffxVghVVA_ZfbMpXDRNGaCUgFUJXtwndVI0 2807
llmebench/tasks/STS.py sha256=cFbo5uZ2L84qeg_HmzsfaEKXlLkPThbbau2jt6hyEsg 586
llmebench/tasks/Sarcasm.py sha256=6-cSYnzK9VA9Ol3FAfCLQeRCx8qOTSRxhRu2Q-j8dIw 554
llmebench/tasks/Sentiment.py sha256=DRAvV3YVxQ9Ja_GW-JhiIse3qf0wWw1gLMi9NyJLMYc 1058
llmebench/tasks/Spam.py sha256=Hx0a1xLiYG3HFq-D3NDeOyau2BkGjYHocypMkQqpEwQ 484
llmebench/tasks/Stance.py sha256=uWPoBhTC43Yf833q2LWNwHRIZOlxiKCb4d3UL-tltRQ 489
llmebench/tasks/Subjectivity.py sha256=-lpFYSWefgG_TR9IgIFuXZ8llusuP-6BCS_ZQVyOJ1I 1662
llmebench/tasks/XNLI.py sha256=7QoIOvrkPFLhdGIQAZkelnpoRnERwEQhqoLTYsbF0hU 499
llmebench/tasks/__init__.py sha256=xvacnYrqx3Bb1stSe48SGW5pEl0eFddg_C7V9oa2AH0 1648
llmebench/tasks/task_base.py sha256=q0F5floUQN0eYVlZ9PBx2Q42Mo4x1ALWmxdvpGeDBqA 3284
llmebench-1.0.1.dist-info/METADATA sha256=4QtF19WyAF73OC-tBjFh-hfNtsFLW4a5KaxpZDw9a4A 12303
llmebench-1.0.1.dist-info/WHEEL sha256=R0nc6qTxuoLk7ShA2_Y-UWkN8ZdfDBG2B6Eqpz2WXbs 91
llmebench-1.0.1.dist-info/top_level.txt sha256=c3ePXkwAA6QCGCUn3Jce8xlABxNu_ORoc9cuwrfCqF0 10
llmebench-1.0.1.dist-info/RECORD — —

top_level.txt

llmebench