codeparrot-clean

CodeParrot 🦜 Dataset Cleaned What is it? A dataset of Python files from Github. This is the deduplicated version of the codeparrot. Processing The original dataset contains a lot of duplicated and noisy data. Therefore, the dataset was cleaned with the following steps: Deduplication Remove exact matches Filtering Average line length 0.25 Remove auto-generated files (keyword search) For… See the full description on the dataset page:

Source: Hugging Face Hub (codeparrot/codeparrot-clean). Metadata imported from the dataset’s Hub tags.