Wheelodex » Projects » megatron-energon

megatron-energon

View on PyPI — Reverse Dependencies (1)

5.0.0 megatron_energon-5.0.0-py3-none-any.whl

Wheel Details

Project: megatron-energon
Version: 5.0.0
Filename: megatron_energon-5.0.0-py3-none-any.whl
Download: [link]
Size: 173399
MD5: 2533553c2913597474a55f3281246795
SHA256: ca8f29992ed70b31d8fb10bd5bff2438fa86bebe907a3f8394fe212edcb25fb4
Uploaded: 2025-01-17 08:44:02 +0000

dist-info

METADATA · WHEEL · RECORD · entry_points.txt

METADATA

Metadata-Version: 2.4
Name: megatron-energon
Version: 5.0.0
Summary: Megatron's multi-modal data loader
Author-Email: Lukas Vögtle <lvoegtle[at]nvidia.com>, Philipp Fischer <pfischer[at]nvidia.com>
Project-Url: Homepage, https://github.com/NVIDIA/Megatron-Energon
Classifier: Operating System :: OS Independent
Classifier: Programming Language :: Python :: 3
Classifier: Programming Language :: Python :: 3.9
Classifier: Programming Language :: Python :: 3.10
Classifier: Programming Language :: Python :: 3.11
Classifier: Programming Language :: Python :: 3.12
Requires-Python: >=3.8
Requires-Dist: braceexpand
Requires-Dist: click
Requires-Dist: numpy
Requires-Dist: pillow (>=10.0.1)
Requires-Dist: pyyaml
Requires-Dist: s3fs
Requires-Dist: torch
Requires-Dist: tqdm
Requires-Dist: webdataset
Requires-Dist: black; extra == "dev"
Requires-Dist: isort; extra == "dev"
Requires-Dist: myst-parser; extra == "dev"
Requires-Dist: sphinx; extra == "dev"
Requires-Dist: sphinx-click; extra == "dev"
Requires-Dist: sphinx-rtd-theme; extra == "dev"
Requires-Dist: sphinxcontrib-napoleon; extra == "dev"
Requires-Dist: torchvision; extra == "transforms"
Provides-Extra: dev
Provides-Extra: transforms
Description-Content-Type: text/markdown
License-Expression: BSD-3-Clause
License-File: LICENSE
[Description omitted; length: 3733 characters]

WHEEL

Wheel-Version: 1.0
Generator: hatchling 1.27.0
Root-Is-Purelib: true
Tag: py3-none-any

RECORD

Path Digest Size
megatron/energon/__init__.py sha256=fuAI7ya74nKuwZ3aqLRXge4PYKfK0NYsv1jk5i0w38w 3702
megatron/energon/dataset_config.py sha256=5aSUQTWxycxBOn7UZOkGRb8QLBDpCDLcp2_vo4bfz4g 4077
megatron/energon/errors.py sha256=_Ax825JRq8mNuLzsLOd-RV-joEhAcBdIq-rmKI9GryA 2647
megatron/energon/loader.py sha256=dsWugHnMkuF41bY0-_2SRgG789-6dZjy8fp4aum88os 3325
megatron/energon/module_loader.py sha256=XuHcnYc7M4YcjEflroMeNDzsSAKGBjF853oWeYs4Gdk 2359
megatron/energon/retry_stream.py sha256=Dg43fboxY-VCrJByOG11XREXmbkgp6MFO_XiBylZbSg 2552
megatron/energon/rng.py sha256=Pbux0wJRqSGPrRR0dmA6X46ZaU0gL-Y8Gi0eAqyw93s 6411
megatron/energon/savable_loader.py sha256=P1sFABju2RqH9EdhPv5FTLt63MUeMFWDX6Vp6juV7Xk 55641
megatron/energon/typed_converter.py sha256=aBNIiq8-MvUc0QbCazKZ0pAJ2Yt6pKwja9a_cpiOiA0 41595
megatron/energon/worker.py sha256=CivnTkq2ZDgBYhS4xmSf7Gee4OcSeohwfx43eUeUKd8 11763
megatron/energon/cli/__init__.py sha256=4Pnukgu-f97lWqk45kATfrBcKZeOm8nMf438lbvVRwo 82
megatron/energon/cli/main.py sha256=KhT1owAtw98gl3e5-I5OQ-xyI3UZ10k_GVVhQ5BeSf8 1150
megatron/energon/epathlib/__init__.py sha256=RIZJk3x2_N-RvhdPoVw0DdQRQO76IEvkLiXQI-B4NkY 154
megatron/energon/epathlib/epath.py sha256=wG3BvG1Cj_lcrQ8t3HeLgdx3mjv2An_FQLT0O2ry0YE 13397
megatron/energon/epathlib/rclone_config.py sha256=-GsSPZ-uvULE4lhUUlIE_RMvjTW9W-Vl0z1Ea8WCzQ8 2996
megatron/energon/flavors/__init__.py sha256=c6E1LkhSm-w4ktkRR3_XwFp8Vpza_HJyiEhUAEKA8no 2404
megatron/energon/flavors/base_dataset.py sha256=zOydwAVyKaHpFvOTUc_aMLMPZUhTnJvoo4DU-jldNHw 16090
megatron/energon/flavors/captioning.py sha256=c73p1a3f_iKaCvOQeSK2xOj1moy1tcFQ4-jXc6SrON8 1255
megatron/energon/flavors/crude.py sha256=6keRQlVyPk7yuc1SDOWOq231o8LThdAAes7CyIxvx00 2212
megatron/energon/flavors/image.py sha256=KNJRV85qdLB7D7UVWbG9Vqani67Lp0pb2C6TBaUEiWM 1214
megatron/energon/flavors/image_classification.py sha256=_dTltGMYXFaNOG_KRbiRWwQeY6OiCMVLsNsZNaKEPgg 1420
megatron/energon/flavors/interleaved.py sha256=qPYJxlFhqsIDNmEOkhjSF2xuk0_DC1O4O9PjLQHlCm4 1321
megatron/energon/flavors/multichoice_vqa.py sha256=MIe6BPZ5JWFbzmc2qIz-2ocu3QhPnT3N7VvM06D76DU 1472
megatron/energon/flavors/ocr.py sha256=A79FCGRssMPqO871HDE-38xI_k9shqCST8yi6rV_mIM 2323
megatron/energon/flavors/similarity_interleaved.py sha256=R31-WXm05OvPcceC0Nf5Tog4nIqdidHg2qk8nsp9vOY 1723
megatron/energon/flavors/text.py sha256=I8pDqgAE-q_Sgoz15KkoEPPVmCziL_nMgySkzGR5ZrQ 1135
megatron/energon/flavors/vid_qa.py sha256=PR1r9OmQpVqZzXcAwjkCSR73ByaFJWKKONoHdbXBnEw 1513
megatron/energon/flavors/vqa.py sha256=PXl63GyzYzjfIhCtv-K4_SpM4-SfBBTIDZ3q-FoIgTo 1492
megatron/energon/flavors/vqa_and_ocr.py sha256=k5vzYUuu3ymu2njhu2HzGy0H1CjvgAOxMSMQygwztHU 1815
megatron/energon/flavors/webdataset/__init__.py sha256=5qP2QHJ_lSFJZYNMqL9si415QiIcdNAgTNudt0JM4V0 1196
megatron/energon/flavors/webdataset/base_webdataset.py sha256=aXgUg7v6JXyTHJM0VRLVgQ-7olELU1rmq9je735QvhY 7711
megatron/energon/flavors/webdataset/config.py sha256=JW_cavR_qTrqMDvXUf6Mb7yzg6FROHZEcy939mH1-BI 228
megatron/energon/flavors/webdataset/default_decoder_webdataset.py sha256=RVd032QSJHHc7_Gp_p5kr7WynB9oK6fNEzkWd9JZphs 3643
megatron/energon/flavors/webdataset/default_generic_webdataset.py sha256=WWpQ4gX7vATrMDnLbraEu0yzcXg18cFThfgbvkKaGks 5226
megatron/energon/flavors/webdataset/empty_dataset_error.py sha256=8ao1gsO8jVZcPotu2O9aiv5xuCpevHSVqPQNh9l3ues 162
megatron/energon/flavors/webdataset/error_handler.py sha256=_-wXQpbJCvSlnFHwzU38tzFEvv5ClNoRxTnggcV_f5U 1615
megatron/energon/flavors/webdataset/field_access.py sha256=qWlYabJySofP9T45oylIRkyne58zEYKdxP4zkgfvkuI 2651
megatron/energon/flavors/webdataset/itar.py sha256=GzDp7wrTS6kwnECxLQxPHX9niCWpjTqG6ZwRslr9SCc 4745
megatron/energon/flavors/webdataset/joined_webdataset.py sha256=lyvAYNzIeR2B-gnSwPScQeUIU0rf93RoiAlvbWL1Rfo 10904
megatron/energon/flavors/webdataset/metadata.py sha256=ioI5jSIt6yRTkVXTsrARN8-ZNp3vCe0FXbYWJTEP7ec 2519
megatron/energon/flavors/webdataset/prepare.py sha256=kgnkVytSuxLOPQBqEeiAqS7BdVVFNcKNbKiIcr_sqVQ 9893
megatron/energon/flavors/webdataset/sample_loader.py sha256=OK1l8vX4xUjGzL4x7yyeNhNJsapigQLS9eKDmp1Zc4w 36178
megatron/energon/flavors/webdataset/sharder.py sha256=fBc8huw4Ic5ETpMMtzQ4yYM2QeepBjbVVmzsSHms5yE 12941
megatron/energon/flavors/webdataset/single_shard_reader.py sha256=gd-bcBR19EDUWtqd-ggsDtOqI8PiXzcHSMPpsyXekcE 10966
megatron/energon/flavors/webdataset/standard_webdataset.py sha256=q209YjLRdqYAQQsuBoGshrphLQr6BK-bzt0Da8fIrO8 3132
megatron/energon/flavors/webdataset/structs.py sha256=FewH0uKm5-ZM2z4_G8bgvfNQLdGFrMYHecq4wXGmpZ0 2302
megatron/energon/flavors/webdataset/video_data.py sha256=71q1r0CpUWU4t13NHe9ajhQkpJ0d7ZYhIHwVrJ6l3Ic 541
megatron/energon/metadataset/__init__.py sha256=PigR05gygObBCaM9J3Ht2W4qdbPMIcf3rBNWNQ2G5L8 556
megatron/energon/metadataset/dataset_loader.py sha256=U-yOtW81lCUnjQiGFaPHDaHeBF0Lj-B99RYyE_MhXkA 3753
megatron/energon/metadataset/join_dataset_loader.py sha256=olXzaMfysN2VyedTdWkkIq_nVEP37WqBKnuaeFlss7A 5223
megatron/energon/metadataset/loader.py sha256=7g0HrQkzRlu59bYnBaQU5NEIp_8gMCDZlv8ATRcK40k 1103
megatron/energon/metadataset/loader_interface.py sha256=NHFYLCDTvQuUJC5PqIRn3evOI0hcSGM9LijxRSwIav0 2711
megatron/energon/metadataset/metadataset.py sha256=7v74D-gItYjXCb_px6x_qmGw3ixT6Stwgs9ZF5E4ucM 6239
megatron/energon/metadataset/metadataset_v2.py sha256=EqR62jslUmqC7Zw4WxQWSWMw3bCikzHCZutjSH-bviM 12728
megatron/energon/task_encoder/__init__.py sha256=UHGy1ZLelDn0enXkvsiXo4WTioiooffFfWumxU9eX3I 686
megatron/energon/task_encoder/base.py sha256=rwYCR5GZkCsYluqcv5vJrwZ6TeIg3HkV67LoCT1GL3Y 31196
megatron/energon/task_encoder/cooking.py sha256=YgLl3_xKqajExHvtnpy6n8xCJabUWApL6Zs_WDuZIoM 2053
megatron/energon/task_encoder/loader.py sha256=mqH_g15trS225elZi9bwPetBdpIqizCHYik3Tg0Hrm0 9142
megatron/energon/tools/__init__.py sha256=4Pnukgu-f97lWqk45kATfrBcKZeOm8nMf438lbvVRwo 82
megatron/energon/tools/analyze_debug.py sha256=FZx39PnckrD4tTvFtfoIMETmrWbivZfmbQbx8pivQrc 19767
megatron/energon/tools/info.py sha256=eJsqxp5Nl_vtTmm4I7w6fMhgwK3Qgbgp6_AEyYepwoE 3529
megatron/energon/tools/lint.py sha256=HydGmI0YRqMVTSpy6CtftSmKMluHXneCGcmnuaXM72Q 3884
megatron/energon/tools/prepare.py sha256=qHHeZXch1oGI0ZFC8T56plgAQmaJtWXccC5U_-CdUyQ 14247
megatron/energon/tools/preview.py sha256=9puLU5_DsnyeA-Yfjhs7QkX_vQMUobHlImwyKOPjkBw 5312
megatron/energon/transforms/__init__.py sha256=LNP93Aszhythh-cZAiMP8TQkoTASBQUI1TH_jsrv98Q 795
megatron/energon/transforms/common.py sha256=27JBr6Ou_Zhz3c2POm54cMibB069GxJAhK0tK87cIMo 312
megatron/energon/transforms/custom.py sha256=WG7an6Et_uBL3ul2wBfxcIz24r9vYCiHO57cixt3DBA 347
megatron/energon/transforms/mappers.py sha256=AbMvwMbZiNQbzBZD68XMac2rVjnyIDbgrln5VVkjqFQ 15081
megatron/energon/transforms/merge.py sha256=QVgIFBeYSXaxrDqaKM2dOGXWJHyO-XwCrIDRZ3so_PM 6821
megatron/energon/wrappers/__init__.py sha256=Ec8H5genvHSN8luKsNVmYEI_ngxg-5MNgsEhC-K4pH4 1668
megatron/energon/wrappers/_log_exception.py sha256=_HWfFBXcXNlyd2gAYvfHnaEC9ZI6ramAZ1ldeuGP3Uw 363
megatron/energon/wrappers/base.py sha256=Zvdqz45DHCNYj1msgNGeLh4shW-z3AZbcz0S57Ohri0 7310
megatron/energon/wrappers/batch_dataset.py sha256=Vsho2S0O6867b_-cU3qYTI_obbCX7nlaLpY3ftw4YDU 13922
megatron/energon/wrappers/blend_dataset.py sha256=cEKXDHrEdplGiR3kANREukUny0VQTHhGH3D56q3rJQE 6151
megatron/energon/wrappers/buffer.py sha256=3cUD-Mtuz4bsqX35wpXA3Q3K3gvnH3GcBCdxsukh6aI 6582
megatron/energon/wrappers/concat_dataset.py sha256=iVebM-XGOzlCIACjAabOBSMJ0NzTUIFjp17tVUm71tQ 3904
megatron/energon/wrappers/epochize_dataset.py sha256=-9FUzbiT9ZOQ0cMTVIGoGAhxrre4nnC_mm2g3PXCCyw 5048
megatron/energon/wrappers/filter_dataset.py sha256=vMnFBOWk9bAee6TjpNpbin3AmjpPgYg36VX3hBX53BY 4005
megatron/energon/wrappers/gc_dataset.py sha256=07JX9J7IYdA3zKvQm43X1tm8goDtqngUHW9m9obWItY 4874
megatron/energon/wrappers/group_batch_dataset.py sha256=jXqRFoSDXuYRZoSWwVnzmK_HDeLrA45guahafBUxM5Y 13276
megatron/energon/wrappers/iter_map_dataset.py sha256=v_WIhp_ZhU3nJ8fk4WIWTK3e6tMdOzYYGdMKychtXgw 9502
megatron/energon/wrappers/limit_dataset.py sha256=PnKJJwL9S5bkLt6_W-oZoSfwF4O_RxUSR19gNzCwQWA 5021
megatron/energon/wrappers/log_sample_dataset.py sha256=nGnbAkDO9oa_CwMthdsD4uz-J6oyCn8Tnw1obsbggw0 4608
megatron/energon/wrappers/map_dataset.py sha256=3wF8rjNR3E0yuqR2LnVUkbgVbW83qMkCnKFHlW9YZN0 11772
megatron/energon/wrappers/mix_batch_dataset.py sha256=v_JbtgC6pqnE4oClw3i_EofiOaPqHx5Ev5X6HxLau-8 5097
megatron/energon/wrappers/packing_dataset.py sha256=uRvDoR06THtid8d-48MlWQyr31UiW1h2QWjONo8frd4 16685
megatron/energon/wrappers/repeat_dataset.py sha256=M5MW7unUzpjF2f2aNLOY168TmaO2HvXtgmurXJEKPIY 4136
megatron/energon/wrappers/shuffle_buffer_dataset.py sha256=SWtKPu2J8AG9L-1jHslEOlABo_G44dk8Ax_Lj7fMYhw 4020
megatron/energon/wrappers/skip.py sha256=d9-Y6IjdnVrk3SK8A4v6aCpud13bAYjyCbLwM3nI6bM 174
megatron_energon-5.0.0.dist-info/METADATA sha256=o1o7UMvhSj0OIdvV-OvCk4Fjz2XR-ash0XtwoMBSh2o 5016
megatron_energon-5.0.0.dist-info/WHEEL sha256=qtCwoSJWgHk21S1Kb4ihdzI2rlJ1ZKaIurTj_ngOhyQ 87
megatron_energon-5.0.0.dist-info/entry_points.txt sha256=ck6KlIK7RuLNOKzeVpEbsXDn_uEo-xWYJnLwzZIOFgc 59
megatron_energon-5.0.0.dist-info/licenses/LICENSE sha256=cdcquNTkyUXAI0Hq9htSPdDh7HWTUspiY8NdxGce2hQ 25793
megatron_energon-5.0.0.dist-info/RECORD — —

entry_points.txt

[console_scripts]
energon = megatron.energon.cli.main:main