dclm-baseline-1.0-parquet

DCLM-baseline Note: this is an identical copy of where all the files have been mapped to a parquet format. DCLM-baseline is a 4T token / 3B document pretraining dataset that achieves strong performance on language model benchmarks. Below are comparisions of model trained on DCLM-baseline with other models in the 7B regime. Model Params Tokens Open dataset? CORE MMLU EXTENDED Open weights, closed datasets… See the full description on the dataset page:

Source: Hugging Face Hub (mlfoundations/dclm-baseline-1.0-parquet). Metadata imported from the dataset’s Hub tags.