DOLMino Mix (November 2024)

DOLMino dataset mix for OLMo2 stage 2 annealing training. Mixture of high-quality data used for the second stage of OLMo2 training. Source Sizes Name Category Tokens Bytes (uncompressed) Documents License DCLM HQ Web Pages 752B 4.56TB 606M CC-BY-4.0 Flan HQ Web Pages 17.0B 98.2GB 57.3M ODC-BY Pes2o STEM Papers 58.6B 413GB 38.8M ODC-BY Wiki Encyclopedic 3.7B 16.2GB 6.17M ODC-BY StackExchange CodeText 1.26B 7.72GB 2.48M CC-BY-SA-{2.5, 3.0, 4.0}… See the full description on the dataset page:

Source: Hugging Face Hub (allenai/dolmino-mix-1124). Metadata imported from the dataset’s Hub tags.