Wheelodex » Projects » lddl

lddl

View on PyPI — Reverse Dependencies (0)

0.1.0 lddl-0.1.0-py3-none-any.whl

Wheel Details

Project: lddl
Version: 0.1.0
Filename: lddl-0.1.0-py3-none-any.whl
Download: [link]
Size: 94446
MD5: 03b7626389395417a55e769e18bf081a
SHA256: 4924fc2616e1abc4654b6bd1f0c8fa49fb3582a46e8d01651d9105771cc3fe6f
Uploaded: 2023-06-02 04:14:42 +0000

dist-info

METADATA · WHEEL · RECORD · top_level.txt · entry_points.txt

METADATA

Metadata-Version: 2.1
Name: lddl
Version: 0.1.0
Summary: Language Datasets and Data Loaders for NVIDIA Deep Learning Examples
Author: Shang Wang, Joseph Jennings, Bobby Chen, Shijie Wang, Peter Dykas
Author-Email: jjennings[at]nvidia.com
Home-Page: https://github.com/NVIDIA/lddl
Classifier: Development Status :: 3 - Alpha
Classifier: Programming Language :: Python :: 3 :: Only
Requires-Python: >=3.6
Requires-Dist: dask[complete] (==2021.7.1)
Requires-Dist: distributed (==2021.7.1)
Requires-Dist: dask-mpi (==2021.11.0)
Requires-Dist: bokeh (==2.4.3)
Requires-Dist: pyarrow (>=4.0.1)
Requires-Dist: mpi4py (==3.1.3)
Requires-Dist: transformers (==4.16.2)
Requires-Dist: wikiextractor (==3.0.6)
Requires-Dist: cchardet (==2.1.7)
Requires-Dist: awscli (>=1.22.55)
Requires-Dist: gdown (==4.5.3)
Description-Content-Type: text/markdown
License-File: LICENSE
[Description omitted; length: 21535 characters]

WHEEL

Wheel-Version: 1.0
Generator: bdist_wheel (0.38.4)
Root-Is-Purelib: true
Tag: py3-none-any

RECORD

Path Digest Size
lddl/__init__.py sha256=47DEQpj8HBSa-_TImW-5JCeuQeRkm5NMpJWZG3hSuFU 0
lddl/random.py sha256=pAAPisXUtkoTGMa9zhg2un6Em7pvgfbQJJypsxN9-Qg 2078
lddl/types.py sha256=BejV35P25L6NSag4XgEj_oUcTbT1t_0Jz-GFVWDs9Ww 1444
lddl/utils.py sha256=jGVahQcyvCfgmOoU_z2Kt0pJFxchQgnKzUbiyWYpgZo 3231
lddl/dask/__init__.py sha256=47DEQpj8HBSa-_TImW-5JCeuQeRkm5NMpJWZG3hSuFU 0
lddl/dask/load_balance.py sha256=Txn6cDcOa5fQLqjQ1aTDtcdgawCg1DywjKZLIxfPlDM 14666
lddl/dask/readers.py sha256=Wa2_S6VcATasOqJ34DVWjg_qL3l0fRMPQ_fXfn04la0 3678
lddl/dask/bart/__init__.py sha256=47DEQpj8HBSa-_TImW-5JCeuQeRkm5NMpJWZG3hSuFU 0
lddl/dask/bart/pretrain.py sha256=9M3ft1h3q6r97q1azxZ0hhDzeXbgBnj5a-3HGD-vSpE 8648
lddl/dask/bert/__init__.py sha256=47DEQpj8HBSa-_TImW-5JCeuQeRkm5NMpJWZG3hSuFU 0
lddl/dask/bert/binning.py sha256=UNMl4xcSqrHFhswmNkFGmQTXZqZdZDh9RWNEsk4oo7g 15127
lddl/dask/bert/pretrain.py sha256=Ja7j1RGa2cGUp-6GgVVMlT2pYTvlL8gajwjwyW7Q3Zc 30113
lddl/download/__init__.py sha256=47DEQpj8HBSa-_TImW-5JCeuQeRkm5NMpJWZG3hSuFU 0
lddl/download/books.py sha256=htXWKIZo9xIJnPviWq9Uv9ZphxH7dPpsRxrl--ksplM 7957
lddl/download/common_crawl.py sha256=fw8G5q_cBru-Ac9DguNmHDh1NkMLLgHqmd1VdHQBOL8 17449
lddl/download/openwebtext.py sha256=D5rhXHfgqMYU4Uhs5gf5BF1Zc58g1w-zTYrx6h4CyXg 7073
lddl/download/utils.py sha256=dIhcykazns4Xin0TyWASgNw8OoXG_oL7vBR3AF8LGSA 1977
lddl/download/wikipedia.py sha256=8Tqdbp6kD9Chm5Qq8tXlBEqBjF5o5CESZ0RYCRg8ZgM 9901
lddl/paddle/__init__.py sha256=Cn6O5MkXRWlZXQlbVft-KAaPIHuH1kzMygqXnHqp20g 48
lddl/paddle/bert.py sha256=9lJtV-SU6R4tCiPthztMy7ZW4kNqKbJhiS-qood51eY 16036
lddl/paddle/dataloader.py sha256=06jy0HQqx50ApxfvN0pcL3gyrwCaRWroTJQhhljkqeI 3587
lddl/paddle/datasets.py sha256=bOUdTh03fqQmz9-MtO9TbWijmSSJcD4f5i-gHlEoBTA 10338
lddl/paddle/log.py sha256=zH0HFWW6BJIzh9sF7DUbSnz9jMGh4UM7h7UOf--Dxdw 4395
lddl/paddle/utils.py sha256=Gv_-jkeEkf3Lq0YtaSe6gmBZQQiSyRqF6E8k0q2Ctk4 4344
lddl/torch/__init__.py sha256=Cn6O5MkXRWlZXQlbVft-KAaPIHuH1kzMygqXnHqp20g 48
lddl/torch/bert.py sha256=Eg1AOG_UhSAeBbLOXnHwIzUPNvnSRdrM0AnXHF3V98U 15932
lddl/torch/dataloader.py sha256=xMUiGGXYqBp-PV54us3YnNT_l8QtB8oqxMJQc_noCFE 3593
lddl/torch/datasets.py sha256=EFg1KzzPjYfPGUiOwaOypXYIC3JisLHvix-Czpgx5Dk 10191
lddl/torch/log.py sha256=F9OeUoX5fX8fDk9dAqLnt2jmnZoszwAnqsMeqE3uc6s 4338
lddl/torch/utils.py sha256=YnRLHrnJHXRAxU2C94ahcjinTnRRXOvodqsU8HgVOIc 3225
lddl/torch_mp/__init__.py sha256=Cn6O5MkXRWlZXQlbVft-KAaPIHuH1kzMygqXnHqp20g 48
lddl/torch_mp/bert.py sha256=GGg5rV5VyJsPdQsiJp21w7wOteub44sXGLoT8kOHwQI 18821
lddl/torch_mp/dataloader.py sha256=o3-j6naNm1nxQoQugSEAl-bXZy9lctfcfnboiFJPnO8 5021
lddl/torch_mp/datasets.py sha256=_c_mQ5awevamMe0jNcrfdMgxYlcX_JsvdF005ZfN2bo 10738
lddl/torch_mp/log.py sha256=F9OeUoX5fX8fDk9dAqLnt2jmnZoszwAnqsMeqE3uc6s 4338
lddl/torch_mp/utils.py sha256=-F2EW7cIPbJ7BDlN_6qK-n59Xwz0Glz5FCuuYLo776U 3749
lddl-0.1.0.dist-info/LICENSE sha256=ACk09lsF8W0WWo-00HKak13Dit_N2Am0sXlUHWZ8fjQ 1150
lddl-0.1.0.dist-info/METADATA sha256=rbr9jCX2m7MGW9IIiSa7y6kgS2VBr-qBPJBf5zs9k6o 22392
lddl-0.1.0.dist-info/WHEEL sha256=2wepM1nk4DS4eFpYrW1TTqPcoGNfHhhO_i5m4cOimbo 92
lddl-0.1.0.dist-info/entry_points.txt sha256=6kSq7adSZJl-pysePPaVYb5n0ELHgREjt2bdwjQDoyo 532
lddl-0.1.0.dist-info/top_level.txt sha256=eb8XlV4-nF_trVAFzTPWkLcAlF2lhSTrdADbDIb1xk0 5
lddl-0.1.0.dist-info/RECORD — —

top_level.txt

lddl

entry_points.txt

[console_scripts]
balance_dask_output = lddl.dask.load_balance:console_script
download_books = lddl.download.books:console_script
download_common_crawl = lddl.download.common_crawl:console_script
download_open_webtext = lddl.download.openwebtext:console_script
download_wikipedia = lddl.download.wikipedia:console_script
generate_num_samples_cache = lddl.dask.load_balance:generate_num_samples_cache
preprocess_bart_pretrain = lddl.dask.bart.pretrain:console_script
preprocess_bert_pretrain = lddl.dask.bert.pretrain:console_script