datatrove: Build Large-Scale Text Data Pipelines
DataTrove is a Python library for processing, filtering and deduplicating text at large scale with reusable pipeline blocks. Its platform-agnostic executors run locally or on Slurm, while fsspec supports local, remote and cloud storage.
ActiveHealth 60/100
