Wheelodex » Projects » stopes

stopes

View on PyPI — Reverse Dependencies (0)

1.0.1 stopes-1.0.1-py3-none-any.whl

Wheel Details

Project: stopes
Version: 1.0.1
Filename: stopes-1.0.1-py3-none-any.whl
Download: [link]
Size: 202741
MD5: f6183aa32ca47411e968573d72c73020
SHA256: b8646f70af05000617294bc9dc0ee8e187a31e451e1ff05933c53b899cc7767f
Uploaded: 2022-07-06 15:54:15 +0000

dist-info

METADATA · WHEEL · RECORD

METADATA

Metadata-Version: 2.1
Name: stopes
Version: 1.0.1
Summary: Large-Scale Translation Data Mining.
Author: Facebook AI Research
Project-Url: Source, https://github.com/facebookresearch/stopes
Project-Url: Tracker, https://github.com/facebookresearch/stopes/issues
Classifier: License :: OSI Approved :: MIT License
Classifier: Topic :: Scientific/Engineering
Classifier: Development Status :: 4 - Beta
Requires-Python: >=3.8
Requires-Dist: hydra-core (>=1.2.0)
Requires-Dist: submitit (>=1.4.2)
Requires-Dist: tqdm
Requires-Dist: joblib
Requires-Dist: pytest (>=4.3.0); extra == "dev"
Requires-Dist: pytest-asyncio (>=0.15.0); extra == "dev"
Requires-Dist: pytest-cov (>=2.6.1); extra == "dev"
Requires-Dist: coverage[toml] (>=5.1); extra == "dev"
Requires-Dist: black (==22.3.0); extra == "dev"
Requires-Dist: isort (>=5.10.1); extra == "dev"
Requires-Dist: mypy (>=0.782); extra == "dev"
Requires-Dist: types-emoji; extra == "dev"
Requires-Dist: pylint (>=2.8.0); extra == "dev"
Requires-Dist: flit (>=3.5.1); extra == "dev"
Requires-Dist: fairscale; extra == "mining"
Requires-Dist: faiss-gpu; extra == "mining"
Requires-Dist: sentencepiece; extra == "mining"
Requires-Dist: numpy; extra == "mining"
Requires-Dist: xxhash; extra == "mono"
Requires-Dist: fasttext; extra == "mono"
Requires-Dist: sentence_splitter; extra == "mono"
Requires-Dist: sentencepiece; extra == "mono"
Requires-Dist: indic-nlp-library; extra == "mono"
Requires-Dist: emoji; extra == "mono"
Provides-Extra: dev
Provides-Extra: mining
Provides-Extra: mono
Description-Content-Type: text/markdown
[Description omitted; length: 4892 characters]

WHEEL

Wheel-Version: 1.0
Generator: flit 3.7.1
Root-Is-Purelib: true
Tag: py3-none-any

RECORD

Path Digest Size
stopes/__init__.py sha256=FofbB7f41b9p_IGaChgd5S4iKM9gpn_BHrSeSZB8UGE 626
stopes/core/__init__.py sha256=1yNLkPfdYnLJ9jjXet7MMODpatALPdcTWe_c_EJDAIQ 363
stopes/core/launcher.py sha256=XLHTelt4eF9gRFhghBuNrkSgZxGgZH1P3AsZ7zeBkso 17697
stopes/core/stopes_module.py sha256=EFsYUr38zl0aAwB1XADrA2YhEgiUag2XelLhtEXtRVI 6940
stopes/core/utils.py sha256=BMIBVED80FI7dyZxGQ8doH9dIlImiekj5yxy-5849pQ 6849
stopes/core/jobs_registry/__init__.py sha256=PLYmdsP61ANRZnOy6ZzbbLwLcfgUw7t_VRv6HoSV4GI 198
stopes/core/jobs_registry/registry.py sha256=QEII61KRgOSHhjDqdnpMP0GXb4hWx_jWcPzXJGy4tFc 9886
stopes/core/jobs_registry/stopes_job.py sha256=uCbrnhV4nGxR2m0dEzhKvYMYcuLxdmGsQHikn_nDLR8 8251
stopes/core/jobs_registry/submitit_slurm_job.py sha256=bP05C7W4JcgR0A7ThqHXfyAFWg74OBkryCEW2H_BZ4s 16091
stopes/core/tests/__init__.py sha256=PLYmdsP61ANRZnOy6ZzbbLwLcfgUw7t_VRv6HoSV4GI 198
stopes/core/tests/hello_world.py sha256=fxm0uLDKe52h2B1CK7dI5x6AgoR4dgdAzoqBKMlB4P0 2433
stopes/core/tests/hello_world_array_module.py sha256=Lc1LCOoJIuhJDVAkc8lMmrW9EIcAfNlVw8ukfBlgCgQ 1390
stopes/core/tests/test_registry.py sha256=5F_R6XtrWdpfgbEZDyouMj8bPahackkGpWvLujiFquY 2722
stopes/modules/__init__.py sha256=PLYmdsP61ANRZnOy6ZzbbLwLcfgUw7t_VRv6HoSV4GI 198
stopes/modules/train_fairseq_module.py sha256=vJt-ntu-Wrhy9La-9go-8h7Iqz1IHGk3DxtQWrTbIOY 3540
stopes/modules/bitext/__init__.py sha256=PLYmdsP61ANRZnOy6ZzbbLwLcfgUw7t_VRv6HoSV4GI 198
stopes/modules/bitext/indexing/__init__.py sha256=oUSxqTEv4wh6N2xitpPBqrC2XmC_d_EJU2AE4gj9-CM 197
stopes/modules/bitext/indexing/merge_faiss_indexes.py sha256=x0OrX_kt2uwPm7uDWLKR-o1CL0Fz_U4XFDdHdzUpP24 4602
stopes/modules/bitext/indexing/populate_faiss_index.py sha256=Qu9MWvMarqbemcOicyyu6XLKXOWXbrcpdPxrNDYdauE 13861
stopes/modules/bitext/indexing/sample_embedding_module.py sha256=sobx8bHYN79z0Yedi59d4tr_2KORa_LlcMozdmyi7Sg 4206
stopes/modules/bitext/indexing/train_faiss_index_module.py sha256=yx-vNSwgn_WawCiFfVZBSZWrzISqd4ISkiDE_OW3h9Q 3138
stopes/modules/bitext/indexing/train_index.py sha256=RWeelLabApxegPutAwTUvA0KUm98R_fqb7EzbXaH_uY 1004
stopes/modules/bitext/mining/__init__.py sha256=PLYmdsP61ANRZnOy6ZzbbLwLcfgUw7t_VRv6HoSV4GI 198
stopes/modules/bitext/mining/calculate_distances.py sha256=bNTbBVwFDE11p-3P7OuW1-_gRXEjyptNw1ueVQ_-z8c 4275
stopes/modules/bitext/mining/calculate_distances_utils.py sha256=0OYt5R3OdTW_15vjzi9HanDwxRO58_GI5uFhYiN8D54 3568
stopes/modules/bitext/mining/mine_bitext_indexes.py sha256=FELuZwTzbRpaf2y89qomLHR_NRw6sLX0ZdrgF_jSKl4 3291
stopes/modules/bitext/mining/mine_bitext_indexes_utils.py sha256=abXHFOKuZiRBs2fXmPHOdYiXGgUeI91Acv-IbqPljcQ 8458
stopes/modules/bitext/mining/mine_bitext_sentences.py sha256=XibmZNHMJoDdxWUZCujz58hrqJqQQ87sUOesMSYI_eU 3436
stopes/modules/bitext/mining/mine_bitext_sentences_utils.py sha256=eiRqOgbrD0XsMGL9Eh8dBWy2oqe9j9hFfNldrwxNDjM 5546
stopes/modules/evaluation/__init__.py sha256=PLYmdsP61ANRZnOy6ZzbbLwLcfgUw7t_VRv6HoSV4GI 198
stopes/modules/evaluation/generate_multi_bleu_detok_module.py sha256=vWL2q3C5CDrwPuqEsq9H6ZcS4qLJNTufpBZkdgVZNGQ 8999
stopes/modules/monolingual/__init__.py sha256=PLYmdsP61ANRZnOy6ZzbbLwLcfgUw7t_VRv6HoSV4GI 198
stopes/modules/monolingual/monolingual_sort_dedup.py sha256=QZVOa6egVgk0ibkbYY0JN9MIe50WXBsx7FXANL4FBUQ 4884
stopes/modules/nmt_bitext_eval_utils/__init__.py sha256=oUSxqTEv4wh6N2xitpPBqrC2XmC_d_EJU2AE4gj9-CM 197
stopes/modules/nmt_bitext_eval_utils/preproc_binarized_mined_utils.py sha256=zRexu68Q7ZzeC7ea_CrEqTaI6E-x_Vu_jzQS-N-_yq4 14934
stopes/modules/preprocess/__init__.py sha256=fIEfgPmEw1Spd5F3lNJHUrA0f-odl-SqFBnsmcahWLQ 437
stopes/modules/preprocess/encode_to_npy.py sha256=GJXZj7g0GZqr2jPz60_mDwu2R7Ci85_3sUVrqJN3zwI 2594
stopes/modules/preprocess/fairseq_binarizer_encoder.py sha256=lwpZMWeWWPqXOCZFL3Sv8ipNhpbw_UTA4_NvqgRJtYY 3321
stopes/modules/preprocess/hf_sentence_encoder.py sha256=kf2LEDHJh-bse_fdao4RwCdtMi9pdJIRIQgRFhUNQ_Y 1995
stopes/modules/preprocess/laser_sentence_encoder.py sha256=Pl5H5LSSOHfg4lIWSM5kcBvgjr3eUuEFPzPs5dUHu7U 13890
stopes/modules/preprocess/line_processor.py sha256=J-dPlZm7oSrBB4_Pe4Q58Y4WiweE6rsKX628zq-JwgI 5467
stopes/modules/preprocess/moses_cli_module.py sha256=F76oLi5hWtvupTtYwpNxASI5cNM36uVC-UuKRa_xWKU 4827
stopes/modules/preprocess/multiproc_line_processor.py sha256=upOK3Teuwy0RkXqrBVYa3NiuCF8TqI8eS98fq6WofPw 10639
stopes/modules/preprocess/preprocess_encode_module.py sha256=JIqQEVHW-9NT2WDQU6YJsxsFwTYBmqvIeFwJc_2g3BE 2111
stopes/modules/preprocess/train_spm.py sha256=bwi91cVZ3fURIjvacybqXl0XBO9y55TxCGgmTMipuj4 4477
stopes/modules/tests/__init__.py sha256=PLYmdsP61ANRZnOy6ZzbbLwLcfgUw7t_VRv6HoSV4GI 198
stopes/modules/tests/test_embedding_utils.py sha256=kIaZ1r0zMJNBGZcqGsJYFubmyBxfxslblBHq9VpEXjk 1288
stopes/modules/tests/test_modules_utils.py sha256=2OujhMxLSjdCfYx9PE8bQmgRcAuKTcCXQdEyMKQfeDo 565
stopes/modules/tests/test_populate_index_port.py sha256=qJwfE2W8ItDEjnYYs-2wUCubkVcNDrejdMB0cWGn03w 5787
stopes/modules/tests/test_split_TSV.py sha256=uXMPOanfhm_w3s0adJ7gR_uv1b0qutd2Qrll1ZPvShM 6163
stopes/modules/tests/test_train_index_port.py sha256=Ym_KpgYvJ3skkfRO_CDPpmK0FRiy119vFGATGxeCdKY 1503
stopes/modules/translation/moses-config.lowercase sha256=PUHOP8-4TuX3dpDMlVTCn63cniDjK3TdEChq4R9fv2c 13764
stopes/pipelines/__init__.py sha256=PLYmdsP61ANRZnOy6ZzbbLwLcfgUw7t_VRv6HoSV4GI 198
stopes/pipelines/bitext/.gitignore sha256=AM6LC4EuGAsO_W4A6k7sKneDDi7hzJF2efonGWiJtsM 8
stopes/pipelines/bitext/DemojizeLineProc.py sha256=_wJ9paGifJt28ow5rMvlX7HlHusI2fPzZSgG8QicOPY 5058
stopes/pipelines/bitext/ExtractMetaLineProc.py sha256=OVFkofBHbnVVMNGWbvrBAAAE32Ko3nCnJPbkvHageYk 7097
stopes/pipelines/bitext/__init__.py sha256=PLYmdsP61ANRZnOy6ZzbbLwLcfgUw7t_VRv6HoSV4GI 198
stopes/pipelines/bitext/bitext_eval.py sha256=b4YNqM7f7yH-3l401lx7lOhbZz1PV7v9eiSEmINjQTU 6029
stopes/pipelines/bitext/dedup_local_and_global.py sha256=kDgrGxsquNrpzzHrMf5VX1wPA2wT0h0tMJyjtsCWPY0 3212
stopes/pipelines/bitext/dedup_single_file.py sha256=NgTAVIsGHi0mIjgMhMPQoDqqvcKtJcUgiWQ9FENOblk 2314
stopes/pipelines/bitext/global_mining_pipeline.py sha256=lpKgksfItsbKFUCREZ8vfURXTDvYMKRwXwwmxnKsDKE 12143
stopes/pipelines/bitext/launch_module.py sha256=Wu7cn_SxX2y59tW2goG1r0Ak3Duf43qVM4D5HVj8QRQ 1630
stopes/pipelines/bitext/nmt_bitext_eval.py sha256=VDOorNJJcZHfKmfC7KdTgQITAyuZ_t9IC0fZ9hptnL4 12332
stopes/pipelines/bitext/requirements.txt sha256=zMWAGt9LaA163WTQACY9RzB-cKCH8mO_1Y3umOo1c0M 222
stopes/pipelines/bitext/shard_and_shuffle.py sha256=HmSuywCK5bqehOimIZu5RBuvN-DG4J_-pOoaC_MW3Rs 6095
stopes/pipelines/bitext/conf/bitext_eval.yaml sha256=3WDMzSt4lNgoRMj8m6CJlJKo1B31sVo4maIqxxgZvA8 617
stopes/pipelines/bitext/conf/dedup_local_and_global.yaml sha256=4lWCFDIXzYzZWKZRvcaC0R_mdiyReS6vILEN_CXATPA 293
stopes/pipelines/bitext/conf/dedup_single_file.yaml sha256=44NmFsfBRMP4wEZ5xx-anLMwMdtYFN71ki6-Py290d4 270
stopes/pipelines/bitext/conf/demojize.yaml sha256=Dh8m83Rm1XIgXVz19Fs9S9kJN6jv3gZxZGiJfsiFDCY 318
stopes/pipelines/bitext/conf/extract_meta.yaml sha256=14hPBl3k7LJZNUhwgnnQEi6D52LD_1jIHihQuqaXAN8 254
stopes/pipelines/bitext/conf/global_mining.yaml sha256=nfnBwyLJZnrHQtPlQLY1AlabUZJpEyTFb0VAJVrSgSg 1603
stopes/pipelines/bitext/conf/launch_conf.yaml sha256=QXK-gfqqtDyrxu1Twgxm_qn-4YsuUlcz7sRe0Vf0VJk 33
stopes/pipelines/bitext/conf/nmt_bitext_eval.yaml sha256=Y9Xr-x_g6ostMJ3Xc9zJ3dbznp7H3owSQ-kyoZ0tWR0 1994
stopes/pipelines/bitext/conf/shard_and_shuffle.yaml sha256=j9ZSfLrUGHfwXjo_EkV5aNgsb8TywKG2Spjrna0V9oU 183
stopes/pipelines/bitext/conf/binarize/fairseq_binarize.yaml sha256=GFgIzelU_et12E2JL4CMx7hndqYxfoj4X2Dij7BwxPA 768
stopes/pipelines/bitext/conf/calculate_distances/base.yaml sha256=hu9A1O8AK8sJ5LFHr7D9EHDZiPdDb4DviSGoyXZvQmE 390
stopes/pipelines/bitext/conf/embed_text/encode.yaml sha256=hX7bQp-Co31sAMdAstwMBwrS5W_5-K60hdVSYhJW3ik 121
stopes/pipelines/bitext/conf/embed_text/hf_labse.yaml sha256=nmb_L1W_SKImxBJ-53dj5VBQauzWU0JxPLRA48WAe1c 116
stopes/pipelines/bitext/conf/embed_text/hf_roberta_large.yaml sha256=opJ9reyBeNN_VGVxvMNDxuW4rwUlTVQpFnwZZSRTAwo 131
stopes/pipelines/bitext/conf/embed_text/huggingface.yaml sha256=D1zj8TpsEtT6JLg4MtZQaj_ENXu1zT1RkohPPH2513k 190
stopes/pipelines/bitext/conf/embed_text/laser2.yaml sha256=FM6euR-KPnssRwjXSBaZtbOD-9PuZMKqveLLC4aPpxY 130
stopes/pipelines/bitext/conf/embed_text/laser3.yaml sha256=DvIAdCA1Clo3klufThhf2r0WLtom9i1vX0eObcZiTG0 87
stopes/pipelines/bitext/conf/embed_text/preproc_and_encode.yaml sha256=0z0UZx_2OBVvn7kWU0CKH0js7c5qk8CUeC2T9S0F22U 287
stopes/pipelines/bitext/conf/embed_text/config/standard_conf.yaml sha256=s6d81l7XJGNNM3R3G4NGyhjrsT68jVcpAGLJ0h-Ly9U 519
stopes/pipelines/bitext/conf/embed_text/config/encoder/hf_encoder.yaml sha256=7PLf8pddMZvhW5uVc7cS772sSwD-pj483ioFgdmpAbg 336
stopes/pipelines/bitext/conf/embed_text/config/encoder/laser2_encoder.yaml sha256=SKOqMNfMFO2FUm8XuBGlLYrZHYYtF4EzAgyxzdTT0Is 321
stopes/pipelines/bitext/conf/embed_text/config/encoder/laser3_encoder.yaml sha256=-ieEnfUYLM7dHPfbfJhv2_8FIK4KbHeDbKD0ddlkYYc 256
stopes/pipelines/bitext/conf/embed_text/config/preprocess/moses.yaml sha256=AKW0v74kMA2r7t-FilQcvHTlPUSvqo8cfErbOH3QPrw 42
stopes/pipelines/bitext/conf/eval/generate_multi_bleu_detok.yaml sha256=oQI-PlgSGNmBp7AcNjOycmV7iOY9VRi52Uf5-G4FXZM 469
stopes/pipelines/bitext/conf/launcher/local.yaml sha256=jQQfItE5wgJ1z0RcVFhViIO7d7Wat61vmG6CG6h2ids 129
stopes/pipelines/bitext/conf/launcher/submitit.yaml sha256=LtXd22O82ROXVf1yb-MOOnBNAKqXE7ZhkcRL1eTZM9Q 128
stopes/pipelines/bitext/conf/launcher/cache/file_cache.yaml sha256=nrNSQJSy9pLL_sgzu2lOdjdru8Y2Pber1tHA-OdcvMA 47
stopes/pipelines/bitext/conf/merge_indexes/merge_faiss.yaml sha256=-GT1Kdm9HQizcrZmevX2nrnegDTW2C0kWwEHWQRMTPI 205
stopes/pipelines/bitext/conf/mine_indexes/base.yaml sha256=x4Bb752_q3Oc1llue9ff_Zrby9mVqClGl-D5sciylc0 481
stopes/pipelines/bitext/conf/mine_sentences/base.yaml sha256=zm9XFVaISwHHV-Bedj2Ispbx_lmnG_L-a4Sbr8-nA0c 446
stopes/pipelines/bitext/conf/moses/standard_conf.yaml sha256=jyWZ6Kgx-BVzXE3CFqr-W0Od3HeoYAOZBmTOzPaA9os 240
stopes/pipelines/bitext/conf/moses_filter/standard_conf.yaml sha256=EolL_JXTf18AzYFRMvkoNXkjl4o6LBc5_s6anjvMJ1k 174
stopes/pipelines/bitext/conf/populate_index/populate_faiss.yaml sha256=fpUc9_pB-zTLtl3qRBCPvOfDegvMbcp7rFJ7nOAjjUM 259
stopes/pipelines/bitext/conf/preproc_binarize_mined/standard_conf.yaml sha256=8v6teAxdqFjEcQIDG4Z7rgrhjwmaVB7fI3zScrqQCrM 455
stopes/pipelines/bitext/conf/preset/demo.yaml sha256=zUsgVdHtYcIZyjMNdNCEQAFrF3ShpI-bNrrLjLTGpAI 4802
stopes/pipelines/bitext/conf/spm/standard_conf.yaml sha256=feAPezOoTwn3QgPuDOUHvoMQPNb0FHWAlQrcwBAdgMk 373
stopes/pipelines/bitext/conf/train_fairseq/nmt.yaml sha256=OMgqi_jFeXThV1ZdQqnqt8eIdhKkcRyuAs7P2StuxnE 934
stopes/pipelines/bitext/conf/train_fairseq/config/params/model/transformer.yaml sha256=HlDAAxHOwJc35yGmLo98pXAEBF4PQ9p4ytMwp0sMTXo 1603
stopes/pipelines/bitext/conf/train_index/train_faiss.yaml sha256=3kOp5Iq86ADVKgjUmDqFv6JFxY9YP-Th5XOZ5T2BY7k 292
stopes/pipelines/bitext/conf/training_after_mining_pipeline/standard_conf.yaml sha256=47DEQpj8HBSa-_TImW-5JCeuQeRkm5NMpJWZG3hSuFU 0
stopes/pipelines/filtering/README.md sha256=dAQwIGTv9IKpcmCinsLgx5QIk2IEH3xxZMCe0lJH3Ao 1749
stopes/pipelines/filtering/__init__.py sha256=47DEQpj8HBSa-_TImW-5JCeuQeRkm5NMpJWZG3hSuFU 0
stopes/pipelines/filtering/configs.py sha256=dmui_W8miRatBvA-RARdaOBccGCH40dzQu0--R-FECs 4122
stopes/pipelines/filtering/dataset.py sha256=qw1fXeTMSKGh7vm43apO2zE0OHtvN8IFIxrA8-YmYeU 3085
stopes/pipelines/filtering/filter.py sha256=pwvO7jlYhMFPte3TXsE-ogemHuBFSEH-pbK4DbqcdT8 8737
stopes/pipelines/filtering/utils.py sha256=pAUU0h0EO6opVt4eI_OHSBcfO5XucOKxi4SVzrVav5Y 562
stopes/pipelines/filtering/conf/example.yaml sha256=Ale4T2YUfghR2C2N9yDohj-qAay9bhiln6185AH9ocM 665
stopes/pipelines/filtering/filters/__init__.py sha256=w6FvbcvqXmS5DlNGD0ppDAMy77Nnzyq2oLnPa5ByCkY 631
stopes/pipelines/filtering/filters/base.py sha256=CaS7mshko2_PcpMD_fbp-9rL9bsVSTMYqRPVGopytAo 2133
stopes/pipelines/filtering/filters/dedup.py sha256=pPDBcQd5MSCHJUK2uJBt4Y8em6OKiHu2QunRtlDsp5Y 2476
stopes/pipelines/filtering/filters/laser.py sha256=61LDQiux4nTbI9W-E7rgBHL9kLZ2t8UVOhjczXdWF0Y 808
stopes/pipelines/filtering/filters/length.py sha256=BjY9Jf-dXX4ZFOGBI-saHYUSDZuflv-lSU5tXFb97D4 3715
stopes/pipelines/filtering/filters/lid.py sha256=cp750V8IFl9MKmf3NeBb9AkNc8OSZ3iN7QaQ-ti5Sgw 2386
stopes/pipelines/filtering/filters/toxicity.py sha256=hKoYgc-qcgLA8xb1_PHPv7nY4I04_u66AKVIMKKBIbA 4504
stopes/pipelines/filtering/scripts/compute_length_factors.py sha256=MbNhegNQ1W1iiBF3zipyBjAuJ2gUgH6KF_jwlnu7gHM 1471
stopes/pipelines/filtering/scripts/populate_data_conf.py sha256=3CrT9436PZ_Z2nL6tPaEpniIgV0Jr4uYMB7MS1fw2nw 5745
stopes/pipelines/monolingual/.gitignore sha256=wCwoK1_Xcwf1vrZpBTCrVCr1iq8eWn4f2HKz-HMCt3k 9
stopes/pipelines/monolingual/README.md sha256=_NfwYZUYyLpmFNbqtEJATmaUL2_iDpUAcpCf-eMpa-k 2537
stopes/pipelines/monolingual/__init__.py sha256=PLYmdsP61ANRZnOy6ZzbbLwLcfgUw7t_VRv6HoSV4GI 198
stopes/pipelines/monolingual/dedup_files.py sha256=wturrC6VapmaVkE5O1DxPIXvT1MjIlKED-c8Ci6uIE8 4955
stopes/pipelines/monolingual/language_equivalences.tsv sha256=iamXsbX35ajC8HL-Z7C6CzKj-3aIWrqDSE1_KwnbA4I 4166
stopes/pipelines/monolingual/language_scripts_200.tsv sha256=EzJwb0EZfGwLmfHrqLAdwZIeW0T2lLYKplhOFXRL0jM 4570
stopes/pipelines/monolingual/monolingual_line_processor.py sha256=CCGe9EUtJ16raLRM8SnO4yp1B-N9isPZ-xDR8a9Wncg 15836
stopes/pipelines/monolingual/monolingual_pipeline.py sha256=K_f5-quhof9znhppwltQWHj6GE4ze_4ahR-6Klvahok 7689
stopes/pipelines/monolingual/requirements.txt sha256=LmO8kcgG37qZlgQpnZAFtTOe5VeFG58KpIA688xgjtU 204
stopes/pipelines/monolingual/conf/dedup_files.yaml sha256=D6AedQTG8hn_XW0HiysMmQverClwBjOp1pzFk2iJwSM 565
stopes/pipelines/monolingual/conf/monolingual.yaml sha256=UYeVowN3yK269r8ZEI9-GelhrzdT168oOKkDBn-ueJo 1393
stopes/pipelines/monolingual/utils/__init__.py sha256=Sf9fm9dRKEvujkiPOSbJZ2NvVCgFx9Zfak4IK2xTbeo 637
stopes/pipelines/monolingual/utils/predict_lid.py sha256=SXupQ9R7zo0lsrpRPXWN1aJ8x0jp_zCnhZK0uECBwwA 2820
stopes/pipelines/monolingual/utils/predict_script.py sha256=VvTqGq7oPIYyxsG4RjR1X0oHDX4NtiQejqwlfzVFEVU 6701
stopes/pipelines/monolingual/utils/remove_non_printing_char.py sha256=FBEFavQNkn3njmU88q4rMw5a0VoLdbH0wgMhC_37Mno 1272
stopes/pipelines/monolingual/utils/remove_regex.py sha256=YxB5b_rJ5sVvXlZZwxXXgC1t1dtH3qUt-5Hz24Si7-0 1826
stopes/pipelines/monolingual/utils/sentence_split.py sha256=3jDDRMzVONcLnkyYvo2_sL6tHO87zwcm_M3gR5JTybo 8008
stopes/pipelines/monolingual/utils/sort.py sha256=M2w7Zai9KblLDgD7ym1C-8KYSkfxmcnKlauLbO8TKo4 1470
stopes/pipelines/monolingual/utils/text_filter.py sha256=1XlI4yEzK_CWT5GYr2BtLZd-e0ONId0MoPx4l1zIcFw 5460
stopes/pipelines/monolingual/utils/text_normalizer.py sha256=WOO-JkU3Eczk5Z1w5NRU6UuTTISeJScQR8Dz-8zY6P8 4946
stopes/pipelines/prepare_data/README.md sha256=T1OZBRzJVt56vaAW_A3KtnrGEvmZya9wesEP3A-l-Eg 5229
stopes/pipelines/prepare_data/__init__.py sha256=oUSxqTEv4wh6N2xitpPBqrC2XmC_d_EJU2AE4gj9-CM 197
stopes/pipelines/prepare_data/cache.py sha256=uov9qvffH-Y7aiuwUswW-cKaAUinPj2Un1_YNNW_wUI 2590
stopes/pipelines/prepare_data/data_types.py sha256=QC_A8l9uGnbAYG04mbIgUriFK-3Lngk6kuinUUiu19s 5886
stopes/pipelines/prepare_data/encode_and_binarize.py sha256=dgguhsTbwHVwgdasAxBOPsMqGjxjooPTo_pJY1qERMQ 6708
stopes/pipelines/prepare_data/prepare_data.py sha256=nO6wu_eShpQrQ_tV1VNbjlAJh-ZXkkUep50te_uSUBw 16841
stopes/pipelines/prepare_data/prepare_vocab.py sha256=tQag58_iv_IpKYStAfzWAbxQK1c1V4A20EzG65Lv7bM 7987
stopes/pipelines/prepare_data/retrieve_data.py sha256=o7DSHEIZCsuapuU8Oc6io8kh9X-uEZFw-C67XrXqC5s 7298
stopes/pipelines/prepare_data/sharding.py sha256=Cb1TbypWR3qP7mqV6SXvleWw8itqji2prwb9saIkdco 2729
stopes/pipelines/prepare_data/spm_tokenizer.py sha256=h7kUayAh4O-kK1ut8VlTwYOzcH0LAoWHimiH-ZpnKWs 2193
stopes/pipelines/prepare_data/utils.py sha256=r3llHEKc6vH4jcmJzJ-kMus6-CxIM_WKLvyDszaPeo0 7826
stopes/pipelines/prepare_data/validation.py sha256=D3YFcKwgvjntQf3o2C0tGzSW9MZG4REebCA75uCB_AU 4742
stopes/utils/__init__.py sha256=PLYmdsP61ANRZnOy6ZzbbLwLcfgUw7t_VRv6HoSV4GI 198
stopes/utils/data_utils.py sha256=YtDQzWuIUrHW8H07xB3cOAnjZ8M_75Vnvh18iR3tyFA 689
stopes/utils/demojizer.py sha256=4Eb-hFZwIuiZJ-npygl2BeIGBSrTWmAweHI6ExTkbCQ 2452
stopes/utils/embedding_utils.py sha256=7lw6-sjR4T9QUn3LR0luku7WRyPn1RhsHAKiTM0WVYk 1721
stopes/utils/map_token_lang.tsv sha256=Bsh9ds-Q5Zb2e989FKDGFlrckAmUpxB3m0-u0hM0r_M 1549
stopes/utils/mining_utils.py sha256=yajVRdWOl5S5mvskLAQAsX3SIkAf8d0o5HiPZql51IA 3333
stopes-1.0.1.dist-info/LICENSE.md sha256=fQ5ZFkFZtCcGnPCK9CT9OgWrshV-1qvnCXQFCjHqWw0 1070
stopes-1.0.1.dist-info/WHEEL sha256=4TfKIB_xu-04bc2iKz6_zFt-gEFEEDU_31HGhqzOCE8 81
stopes-1.0.1.dist-info/METADATA sha256=VkjKBoQvzaxxFJQNCknK-VSKzUC7ATIFKmmcSuuqR88 6452
stopes-1.0.1.dist-info/RECORD — —