FineWeb-Edu 100BT (Shuffled)

FineWeb-Edu 100BT (Shuffled) A globally shuffled version of HuggingFaceFW/fineweb edu 100BT. Part of the Smol-Data collection — tried and tested mixes for strong pretraining. Dataset Description This dataset contains the same ~100B tokens as fineweb edu 100BT but with all documents globally shuffled (seed=42). Use this version when you need randomized document ordering for pretraining. How It Was Created The unshuffled dataset was loaded into memory, shuffled… See the full description on the dataset page: edu 100BT-shuffled.

Source: Hugging Face Hub (HuggingFaceFW/fineweb_edu_100BT-shuffled). Metadata imported from the dataset’s Hub tags.