dedup-rs

View on PyPIReverse Dependencies (0)

0.1.1 dedup_rs-0.1.1-cp39-cp39-manylinux_2_17_x86_64.manylinux2014_x86_64.whl
dedup_rs-0.1.1-cp39-cp39-manylinux_2_17_i686.manylinux2014_i686.whl
dedup_rs-0.1.1-cp39-cp39-manylinux_2_17_armv7l.manylinux2014_armv7l.whl
dedup_rs-0.1.1-cp39-cp39-manylinux_2_17_s390x.manylinux2014_s390x.whl
dedup_rs-0.1.1-cp39-cp39-manylinux_2_17_ppc64le.manylinux2014_ppc64le.whl
dedup_rs-0.1.1-cp39-cp39-manylinux_2_17_aarch64.manylinux2014_aarch64.whl
dedup_rs-0.1.1-cp39-none-win_amd64.whl
dedup_rs-0.1.1-cp39-none-win32.whl
dedup_rs-0.1.1-cp39-cp39-macosx_10_12_x86_64.whl
dedup_rs-0.1.1-cp39-cp39-macosx_11_0_arm64.whl
dedup_rs-0.1.1-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl
dedup_rs-0.1.1-cp38-cp38-manylinux_2_17_i686.manylinux2014_i686.whl
dedup_rs-0.1.1-cp38-cp38-manylinux_2_17_armv7l.manylinux2014_armv7l.whl
dedup_rs-0.1.1-cp38-cp38-manylinux_2_17_s390x.manylinux2014_s390x.whl
dedup_rs-0.1.1-cp38-cp38-manylinux_2_17_ppc64le.manylinux2014_ppc64le.whl
dedup_rs-0.1.1-cp38-cp38-manylinux_2_17_aarch64.manylinux2014_aarch64.whl
dedup_rs-0.1.1-cp38-none-win_amd64.whl
dedup_rs-0.1.1-cp38-none-win32.whl
dedup_rs-0.1.1-cp312-cp312-manylinux_2_17_x86_64.manylinux2014_x86_64.whl
dedup_rs-0.1.1-cp312-cp312-manylinux_2_17_i686.manylinux2014_i686.whl
dedup_rs-0.1.1-cp312-cp312-manylinux_2_17_armv7l.manylinux2014_armv7l.whl
dedup_rs-0.1.1-cp312-cp312-manylinux_2_17_s390x.manylinux2014_s390x.whl
dedup_rs-0.1.1-cp312-cp312-manylinux_2_17_ppc64le.manylinux2014_ppc64le.whl
dedup_rs-0.1.1-cp312-cp312-manylinux_2_17_aarch64.manylinux2014_aarch64.whl
dedup_rs-0.1.1-cp312-none-win_amd64.whl
dedup_rs-0.1.1-cp312-none-win32.whl
dedup_rs-0.1.1-cp312-cp312-macosx_10_12_x86_64.whl
dedup_rs-0.1.1-cp312-cp312-macosx_11_0_arm64.whl
dedup_rs-0.1.1-cp311-cp311-manylinux_2_17_x86_64.manylinux2014_x86_64.whl
dedup_rs-0.1.1-cp311-cp311-manylinux_2_17_i686.manylinux2014_i686.whl
dedup_rs-0.1.1-cp311-cp311-manylinux_2_17_armv7l.manylinux2014_armv7l.whl
dedup_rs-0.1.1-cp311-cp311-manylinux_2_17_s390x.manylinux2014_s390x.whl
dedup_rs-0.1.1-cp311-cp311-manylinux_2_17_ppc64le.manylinux2014_ppc64le.whl
dedup_rs-0.1.1-cp311-cp311-manylinux_2_17_aarch64.manylinux2014_aarch64.whl
dedup_rs-0.1.1-cp311-none-win_amd64.whl
dedup_rs-0.1.1-cp311-none-win32.whl
dedup_rs-0.1.1-cp311-cp311-macosx_10_12_x86_64.whl
dedup_rs-0.1.1-cp311-cp311-macosx_11_0_arm64.whl
dedup_rs-0.1.1-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl
dedup_rs-0.1.1-cp310-cp310-manylinux_2_17_i686.manylinux2014_i686.whl
dedup_rs-0.1.1-cp310-cp310-manylinux_2_17_armv7l.manylinux2014_armv7l.whl
dedup_rs-0.1.1-cp310-cp310-manylinux_2_17_s390x.manylinux2014_s390x.whl
dedup_rs-0.1.1-cp310-cp310-manylinux_2_17_ppc64le.manylinux2014_ppc64le.whl
dedup_rs-0.1.1-cp310-cp310-manylinux_2_17_aarch64.manylinux2014_aarch64.whl
dedup_rs-0.1.1-cp310-none-win_amd64.whl
dedup_rs-0.1.1-cp310-none-win32.whl
dedup_rs-0.1.1-cp310-cp310-macosx_10_12_x86_64.whl
dedup_rs-0.1.1-cp310-cp310-macosx_11_0_arm64.whl
dedup_rs-0.1.1-pp39-pypy39_pp73-manylinux_2_17_x86_64.manylinux2014_x86_64.whl
dedup_rs-0.1.1-pp39-pypy39_pp73-manylinux_2_17_i686.manylinux2014_i686.whl
dedup_rs-0.1.1-pp39-pypy39_pp73-manylinux_2_17_armv7l.manylinux2014_armv7l.whl
dedup_rs-0.1.1-pp39-pypy39_pp73-manylinux_2_17_s390x.manylinux2014_s390x.whl
dedup_rs-0.1.1-pp39-pypy39_pp73-manylinux_2_17_ppc64le.manylinux2014_ppc64le.whl
dedup_rs-0.1.1-pp39-pypy39_pp73-manylinux_2_17_aarch64.manylinux2014_aarch64.whl
dedup_rs-0.1.1-pp38-pypy38_pp73-manylinux_2_17_x86_64.manylinux2014_x86_64.whl
dedup_rs-0.1.1-pp38-pypy38_pp73-manylinux_2_17_i686.manylinux2014_i686.whl
dedup_rs-0.1.1-pp38-pypy38_pp73-manylinux_2_17_armv7l.manylinux2014_armv7l.whl
dedup_rs-0.1.1-pp38-pypy38_pp73-manylinux_2_17_s390x.manylinux2014_s390x.whl
dedup_rs-0.1.1-pp38-pypy38_pp73-manylinux_2_17_ppc64le.manylinux2014_ppc64le.whl
dedup_rs-0.1.1-pp38-pypy38_pp73-manylinux_2_17_aarch64.manylinux2014_aarch64.whl
dedup_rs-0.1.1-pp310-pypy310_pp73-manylinux_2_17_x86_64.manylinux2014_x86_64.whl
dedup_rs-0.1.1-pp310-pypy310_pp73-manylinux_2_17_i686.manylinux2014_i686.whl
dedup_rs-0.1.1-pp310-pypy310_pp73-manylinux_2_17_armv7l.manylinux2014_armv7l.whl
dedup_rs-0.1.1-pp310-pypy310_pp73-manylinux_2_17_s390x.manylinux2014_s390x.whl
dedup_rs-0.1.1-pp310-pypy310_pp73-manylinux_2_17_ppc64le.manylinux2014_ppc64le.whl
dedup_rs-0.1.1-pp310-pypy310_pp73-manylinux_2_17_aarch64.manylinux2014_aarch64.whl

Wheel Details

Project: dedup-rs
Version: 0.1.1
Filename: dedup_rs-0.1.1-cp312-none-win32.whl
Download: [link]
Size: 794206
MD5: 76774031c3274ae59ed072731e72412d
SHA256: ae02a743afb36750f44ea7358588833bc527f5c0850e1b7366c5b18d9872635a
Uploaded: 2024-06-02 06:15:55 +0000

dist-info

METADATA

Metadata-Version: 2.3
Name: dedup-rs
Version: 0.1.1
Summary: A Rust library for deduplication of documents
Author: Wayne Lau,
Classifier: Programming Language :: Rust
Classifier: Programming Language :: Python :: Implementation :: CPython
Classifier: Programming Language :: Python :: Implementation :: PyPy
Requires-Python: >=3.8
Requires-Dist: numpy (>=1.26.4)
Requires-Dist: tqdm (>=4.64.1)
Requires-Dist: datasets (>=2.17.0)
Requires-Dist: scipy (>=1.10.1)
Requires-Dist: xxhash (>=3.0.0)
Requires-Dist: pybloom-live (>=4.0.0)
Requires-Dist: bitarray (>=2.6.2)
Requires-Dist: regex (>=2023.5.5)
Requires-Dist: urllib3 (<=2.0)
Requires-Dist: sphinxcontrib-bibtex (>=2.5.0)
Requires-Dist: zstandard (>=0.21.0)
Requires-Dist: ftfy (>=6.1.1)
Requires-Dist: setuptools (>=69.1.0)
Requires-Dist: psutil (>=5.9.8)
Requires-Dist: fire (~=0.6.0)
Requires-Dist: click (~=8.1.7)
Requires-Dist: click-option-group (~=0.5.6)
Requires-Dist: rich (~=13.7.1)
Requires-Dist: unisim (~=0.0.1)
Requires-Dist: black; extra == "dev"
Requires-Dist: flake8; extra == "dev"
Requires-Dist: isort; extra == "dev"
Requires-Dist: mypy; extra == "dev"
Requires-Dist: pre-commit; extra == "dev"
Requires-Dist: insegel; extra == "dev"
Requires-Dist: pytest; extra == "dev"
Requires-Dist: coverage; extra == "dev"
Requires-Dist: ruff; extra == "dev"
Requires-Dist: tabulate; extra == "dev"
Requires-Dist: scikit-learn; extra == "dev"
Provides-Extra: dev
Description-Content-Type: text/markdown; charset=UTF-8; variant=GFM
License-File: LICENSE
[Description omitted; length: 4850 characters]

WHEEL

Wheel-Version: 1.0
Generator: maturin (1.5.1)
Root-Is-Purelib: false
Tag: cp312-none-win32

RECORD

Path Digest Size
dedup_rs-0.1.1.dist-info/METADATA sha256=0a-ymMy822bQI7we-gL32QozSIZInRMxsoSzWsJqaBU 6465
dedup_rs-0.1.1.dist-info/WHEEL sha256=z8Ql1a7I9FwgD4RlNJNTj6LCRFMzqXuIukXregoopCg 91
dedup_rs-0.1.1.dist-info/license_files/LICENSE sha256=Pd-b5cKP4n2tFDpdx27qJSIq0d1ok0oEcGTlbtL6QMU 11560
text_dedup/ann_unisim.py sha256=6gsp5PTIgoGpybKZ06HNeHOfBsDJpi0oM7aOHWMzz58 7404
text_dedup/bloom_filter.py sha256=IzZAv6N4Bncmq7imljjkJTU9Y36DWexjfTR9X7fVltM 2861
text_dedup/ccnet.py sha256=zEtC_50Xj-fnExnRnep4b8LJUG4g_7RvR-4x8Pkgv50 6407
text_dedup/exact_hash.py sha256=lpUhLtAVJvE34a2iZGkxfNJfw8xMRDEmPTosdP_fLaA 3330
text_dedup/minhash.py sha256=4g_Bisl3GlxfixSn9uPr41fuqYvkGHInU7LH7TiCdsE 12498
text_dedup/minhash_rust.py sha256=DCJdqZbBVQDep-3peGxJ1D-XGMh9XqQOvNsZRo_fULs 4632
text_dedup/minhash_spark.py sha256=t_4K_71XzfrTal6YGAJV-XLP9FTXYVCqwsQVGvfyxQg 17541
text_dedup/simhash.py sha256=2j-QEJqPHyM6oAEaFnxlKInwQgv8YBD-BwBSEiS8MY8 14795
text_dedup/suffix_array.py sha256=hYsJ_bBfs-XHiQA3LPEF0-4QJxqCUqZFMdVQwqysPUg 12452
text_dedup/utils/analysis.py sha256=wAC3abqm8iHhzDjq2WhBUlhqcTZDBmDBKMCuTGlLbCE 3375
text_dedup/utils/args.py sha256=2iV8HSxsR5rsFA_0mVqEfw-lz8XWcj8PJyPjvkXPwAY 15771
text_dedup/utils/const.py sha256=UWl5Jjt_i5JV3f2pkOFrnU7fbhPNObNTyH1ECsfJLD4 60
text_dedup/utils/ftfy_utils.py sha256=I6yFut6JZtWf9zmu84krRaqj3MqOeetrIISKgX94hUU 224
text_dedup/utils/hashfunc.py sha256=eRL1yyiiuY8cX9qud02cw3Mm1L--mgb5TZ1UE85LROY 6593
text_dedup/utils/inspect.py sha256=FJ930ij9PtY0ejyXpjZVs2FbgJQKgzfcurqCxQjoyYc 778
text_dedup/utils/load.py sha256=WBAyDYV9sFm2OheEB7LKfxZGOqsoU_mtBdTFNH5vcWI 1330
text_dedup/utils/memory.py sha256=d_-HmqA3qjdcQWqhe4eZTbWDf1H69XY58B686wnMWfE 396
text_dedup/utils/preprocess.py sha256=xtcA7JZpUONLOHI3ZcbLKWlxjJ3aEopRhXkCzvcvBxY 1498
text_dedup/utils/timer.py sha256=f1lsSD7FYjV5BiKIbopzrPzvqagSHsErj73oLg-eNM0 1798
text_dedup/utils/tokenization.py sha256=CSJTSkep474rGQ95R3PEzZO2J3VGN7MfcuilSumM9_o 1238
text_dedup/utils/union_find.py sha256=qzIYM2u4S3AqpwWWfLQqV06Amm3KNvHFBPIfdghNtc4 3001
text_dedup/utils/__init__.py sha256=vd2KEls7WkbhRfnmmCc6podlGoF0TxD1H5ur5htvkg8 1904
text_dedup/__init__.py sha256=YTinZQ8QErzgZwx2dHXYt4ZTRGIumlE3vCxH-Ienrsw 371
text_dedup/dedup_rs.cp312-win32.pyd sha256=gpDXqlQyYvsN6Uvpe1-wsnb-syPrOEha8ChJ5P1BjDk 1710080
dedup_rs-0.1.1.dist-info/RECORD