dedup-rs

View on PyPIReverse Dependencies (0)

0.1.1 dedup_rs-0.1.1-cp39-cp39-manylinux_2_17_x86_64.manylinux2014_x86_64.whl
dedup_rs-0.1.1-cp39-cp39-manylinux_2_17_i686.manylinux2014_i686.whl
dedup_rs-0.1.1-cp39-cp39-manylinux_2_17_armv7l.manylinux2014_armv7l.whl
dedup_rs-0.1.1-cp39-cp39-manylinux_2_17_s390x.manylinux2014_s390x.whl
dedup_rs-0.1.1-cp39-cp39-manylinux_2_17_ppc64le.manylinux2014_ppc64le.whl
dedup_rs-0.1.1-cp39-cp39-manylinux_2_17_aarch64.manylinux2014_aarch64.whl
dedup_rs-0.1.1-cp39-none-win_amd64.whl
dedup_rs-0.1.1-cp39-none-win32.whl
dedup_rs-0.1.1-cp39-cp39-macosx_10_12_x86_64.whl
dedup_rs-0.1.1-cp39-cp39-macosx_11_0_arm64.whl
dedup_rs-0.1.1-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl
dedup_rs-0.1.1-cp38-cp38-manylinux_2_17_i686.manylinux2014_i686.whl
dedup_rs-0.1.1-cp38-cp38-manylinux_2_17_armv7l.manylinux2014_armv7l.whl
dedup_rs-0.1.1-cp38-cp38-manylinux_2_17_s390x.manylinux2014_s390x.whl
dedup_rs-0.1.1-cp38-cp38-manylinux_2_17_ppc64le.manylinux2014_ppc64le.whl
dedup_rs-0.1.1-cp38-cp38-manylinux_2_17_aarch64.manylinux2014_aarch64.whl
dedup_rs-0.1.1-cp38-none-win_amd64.whl
dedup_rs-0.1.1-cp38-none-win32.whl
dedup_rs-0.1.1-cp312-cp312-manylinux_2_17_x86_64.manylinux2014_x86_64.whl
dedup_rs-0.1.1-cp312-cp312-manylinux_2_17_i686.manylinux2014_i686.whl
dedup_rs-0.1.1-cp312-cp312-manylinux_2_17_armv7l.manylinux2014_armv7l.whl
dedup_rs-0.1.1-cp312-cp312-manylinux_2_17_s390x.manylinux2014_s390x.whl
dedup_rs-0.1.1-cp312-cp312-manylinux_2_17_ppc64le.manylinux2014_ppc64le.whl
dedup_rs-0.1.1-cp312-cp312-manylinux_2_17_aarch64.manylinux2014_aarch64.whl
dedup_rs-0.1.1-cp312-none-win_amd64.whl
dedup_rs-0.1.1-cp312-none-win32.whl
dedup_rs-0.1.1-cp312-cp312-macosx_10_12_x86_64.whl
dedup_rs-0.1.1-cp312-cp312-macosx_11_0_arm64.whl
dedup_rs-0.1.1-cp311-cp311-manylinux_2_17_x86_64.manylinux2014_x86_64.whl
dedup_rs-0.1.1-cp311-cp311-manylinux_2_17_i686.manylinux2014_i686.whl
dedup_rs-0.1.1-cp311-cp311-manylinux_2_17_armv7l.manylinux2014_armv7l.whl
dedup_rs-0.1.1-cp311-cp311-manylinux_2_17_s390x.manylinux2014_s390x.whl
dedup_rs-0.1.1-cp311-cp311-manylinux_2_17_ppc64le.manylinux2014_ppc64le.whl
dedup_rs-0.1.1-cp311-cp311-manylinux_2_17_aarch64.manylinux2014_aarch64.whl
dedup_rs-0.1.1-cp311-none-win_amd64.whl
dedup_rs-0.1.1-cp311-none-win32.whl
dedup_rs-0.1.1-cp311-cp311-macosx_10_12_x86_64.whl
dedup_rs-0.1.1-cp311-cp311-macosx_11_0_arm64.whl
dedup_rs-0.1.1-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl
dedup_rs-0.1.1-cp310-cp310-manylinux_2_17_i686.manylinux2014_i686.whl
dedup_rs-0.1.1-cp310-cp310-manylinux_2_17_armv7l.manylinux2014_armv7l.whl
dedup_rs-0.1.1-cp310-cp310-manylinux_2_17_s390x.manylinux2014_s390x.whl
dedup_rs-0.1.1-cp310-cp310-manylinux_2_17_ppc64le.manylinux2014_ppc64le.whl
dedup_rs-0.1.1-cp310-cp310-manylinux_2_17_aarch64.manylinux2014_aarch64.whl
dedup_rs-0.1.1-cp310-none-win_amd64.whl
dedup_rs-0.1.1-cp310-none-win32.whl
dedup_rs-0.1.1-cp310-cp310-macosx_10_12_x86_64.whl
dedup_rs-0.1.1-cp310-cp310-macosx_11_0_arm64.whl
dedup_rs-0.1.1-pp39-pypy39_pp73-manylinux_2_17_x86_64.manylinux2014_x86_64.whl
dedup_rs-0.1.1-pp39-pypy39_pp73-manylinux_2_17_i686.manylinux2014_i686.whl
dedup_rs-0.1.1-pp39-pypy39_pp73-manylinux_2_17_armv7l.manylinux2014_armv7l.whl
dedup_rs-0.1.1-pp39-pypy39_pp73-manylinux_2_17_s390x.manylinux2014_s390x.whl
dedup_rs-0.1.1-pp39-pypy39_pp73-manylinux_2_17_ppc64le.manylinux2014_ppc64le.whl
dedup_rs-0.1.1-pp39-pypy39_pp73-manylinux_2_17_aarch64.manylinux2014_aarch64.whl
dedup_rs-0.1.1-pp38-pypy38_pp73-manylinux_2_17_x86_64.manylinux2014_x86_64.whl
dedup_rs-0.1.1-pp38-pypy38_pp73-manylinux_2_17_i686.manylinux2014_i686.whl
dedup_rs-0.1.1-pp38-pypy38_pp73-manylinux_2_17_armv7l.manylinux2014_armv7l.whl
dedup_rs-0.1.1-pp38-pypy38_pp73-manylinux_2_17_s390x.manylinux2014_s390x.whl
dedup_rs-0.1.1-pp38-pypy38_pp73-manylinux_2_17_ppc64le.manylinux2014_ppc64le.whl
dedup_rs-0.1.1-pp38-pypy38_pp73-manylinux_2_17_aarch64.manylinux2014_aarch64.whl
dedup_rs-0.1.1-pp310-pypy310_pp73-manylinux_2_17_x86_64.manylinux2014_x86_64.whl
dedup_rs-0.1.1-pp310-pypy310_pp73-manylinux_2_17_i686.manylinux2014_i686.whl
dedup_rs-0.1.1-pp310-pypy310_pp73-manylinux_2_17_armv7l.manylinux2014_armv7l.whl
dedup_rs-0.1.1-pp310-pypy310_pp73-manylinux_2_17_s390x.manylinux2014_s390x.whl
dedup_rs-0.1.1-pp310-pypy310_pp73-manylinux_2_17_ppc64le.manylinux2014_ppc64le.whl
dedup_rs-0.1.1-pp310-pypy310_pp73-manylinux_2_17_aarch64.manylinux2014_aarch64.whl

Wheel Details

Project: dedup-rs
Version: 0.1.1
Filename: dedup_rs-0.1.1-cp311-cp311-manylinux_2_17_ppc64le.manylinux2014_ppc64le.whl
Download: [link]
Size: 1951048
MD5: c2fa6cfa5b9c11fed2f589e4a470071e
SHA256: 5a046ed1349dd1cf30221b0989e4415e3d148b5fba32cfa45e766910586f1f98
Uploaded: 2024-06-02 06:14:36 +0000

dist-info

METADATA

Metadata-Version: 2.3
Name: dedup-rs
Version: 0.1.1
Summary: A Rust library for deduplication of documents
Author: Wayne Lau,
Classifier: Programming Language :: Rust
Classifier: Programming Language :: Python :: Implementation :: CPython
Classifier: Programming Language :: Python :: Implementation :: PyPy
Requires-Python: >=3.8
Requires-Dist: numpy (>=1.26.4)
Requires-Dist: tqdm (>=4.64.1)
Requires-Dist: datasets (>=2.17.0)
Requires-Dist: scipy (>=1.10.1)
Requires-Dist: xxhash (>=3.0.0)
Requires-Dist: pybloom-live (>=4.0.0)
Requires-Dist: bitarray (>=2.6.2)
Requires-Dist: regex (>=2023.5.5)
Requires-Dist: urllib3 (<=2.0)
Requires-Dist: sphinxcontrib-bibtex (>=2.5.0)
Requires-Dist: zstandard (>=0.21.0)
Requires-Dist: ftfy (>=6.1.1)
Requires-Dist: setuptools (>=69.1.0)
Requires-Dist: psutil (>=5.9.8)
Requires-Dist: fire (~=0.6.0)
Requires-Dist: click (~=8.1.7)
Requires-Dist: click-option-group (~=0.5.6)
Requires-Dist: rich (~=13.7.1)
Requires-Dist: unisim (~=0.0.1)
Requires-Dist: black; extra == "dev"
Requires-Dist: flake8; extra == "dev"
Requires-Dist: isort; extra == "dev"
Requires-Dist: mypy; extra == "dev"
Requires-Dist: pre-commit; extra == "dev"
Requires-Dist: insegel; extra == "dev"
Requires-Dist: pytest; extra == "dev"
Requires-Dist: coverage; extra == "dev"
Requires-Dist: ruff; extra == "dev"
Requires-Dist: tabulate; extra == "dev"
Requires-Dist: scikit-learn; extra == "dev"
Provides-Extra: dev
Description-Content-Type: text/markdown; charset=UTF-8; variant=GFM
License-File: LICENSE
[Description omitted; length: 4850 characters]

WHEEL

Wheel-Version: 1.0
Generator: maturin (1.5.1)
Root-Is-Purelib: false
Tag: cp311-cp311-manylinux_2_17_ppc64le.manylinux2014_ppc64le

RECORD

Path Digest Size
dedup_rs-0.1.1.dist-info/METADATA sha256=sK_vz0L50105Qmcw5CtcpWs4glDXnIETAPievpGFZGY 6363
dedup_rs-0.1.1.dist-info/WHEEL sha256=ftUtd1-QOUiDsx-7WNMAK68izpg8CdskP_DFvpOZVhM 131
dedup_rs-0.1.1.dist-info/license_files/LICENSE sha256=z8d0m5b2O9McPEK1xHG_dWgUBT6EfBDz6wA0F7xSPTA 11358
text_dedup/minhash.py sha256=z8dChErPfQXvK7T9c7DvM4oRW2DRXZpXDhrXDcM5geQ 12186
text_dedup/__init__.py sha256=kJYnpIwatWmXK9bDNvSHpPnsv_Ck92vgZkMnzx1EhBk 357
text_dedup/exact_hash.py sha256=HMTGnxvyei_gQQjpfhI-s2tloztrB0RNCm82wmBi0_w 3238
text_dedup/ann_unisim.py sha256=yqKYKrGvgsM3gljyKfb179Hw4DOaSrY6KaSSlBQq0A8 7203
text_dedup/minhash_spark.py sha256=4g973kULpPaGgcn3h2GL-oD4U8j1bfeo8ayQUQRSqLU 16973
text_dedup/utils/load.py sha256=otJWzXM9XdP4s5XWdvyrXdgtF8ZdZqh3zq7xwrJ_2Q0 1286
text_dedup/utils/union_find.py sha256=anNgbtePEU7MntV5e7mFF6ry_RzVd5nq4lNm7-8cL0M 2903
text_dedup/utils/hashfunc.py sha256=Ry1V21TVSaYnfwstTu-a7_0aoqDbF428swCBO3AlkNs 6329
text_dedup/utils/__init__.py sha256=a1D7A9Yhue-3y8-TaYAhcjInLCjxCFtS0idisr4i0q4 1845
text_dedup/utils/analysis.py sha256=P8yyE72MWA92MUR9vQedL5_hsuuW3kM3MYZsYDLW_aI 3257
text_dedup/utils/ftfy_utils.py sha256=LpWCnFXDJnyzQSs2bhxmPtN_8uXuL7LHHR_Nn2diLuY 217
text_dedup/utils/const.py sha256=nayve1kvy5zVZtUViQPShjQJMkBEgk0KWoCfM5wtQus 58
text_dedup/utils/tokenization.py sha256=EdPtz6YdRJjR8_9UnnQBKb7ofSInpFlIQTrbOX0Pskc 1193
text_dedup/utils/memory.py sha256=GnbDz1X3puuD-tPSsfIr_rZq7L4mMFUf_v8a9OY2PhI 380
text_dedup/utils/inspect.py sha256=TgIAL8OLq2LSrnis8X8atM0n0w3P8LQQtjEdBALszHM 753
text_dedup/utils/timer.py sha256=8nZQC8Ju2ypsOpsE69jKKY5smFhsBuKqfMO_vAXkTso 1733
text_dedup/utils/preprocess.py sha256=EBos7nzoNG2oaiUDGtXp73ZpVK_IVy9tC3kRPeewKgM 1439
text_dedup/utils/args.py sha256=1-1panPOpUhjS2v_9sRcviO0l_f4PLHlUO2I4LGlltQ 15303
text_dedup/bloom_filter.py sha256=u27XjsE7bcm54uyID2NMd6ZzLVElj_zLK-Qvq8xIf8s 2775
text_dedup/ccnet.py sha256=oBTUh6mi6wsd7MHNVgAbW08d8hQU04qg_x5_v6XdmPQ 6203
text_dedup/suffix_array.py sha256=PXdVe5IHtInlGaF3MtQecFhrCOy9ma4LEqyvabngFu0 12063
text_dedup/minhash_rust.py sha256=4_EvKBfJ1eiwgT4kHLDReUqZROGywreCHNazDgnUNHQ 4507
text_dedup/simhash.py sha256=dMYQf0T9lK5VC1dSdVjqTzeoAXQ6EisfygKrk4QRakM 14321
text_dedup/dedup_rs.cpython-311-powerpc64le-linux-gnu.so sha256=olP-GPe8--N6u-S7NYK9eLT1rcOTFqaG_lRyD2BvGiQ 7019168
dedup_rs-0.1.1.dist-info/RECORD