mrcr

OpenAI MRCR: Long context multiple needle in a haystack benchmark OpenAI MRCR (Multi-round co-reference resolution) is a long context dataset for benchmarking an LLM’s ability to distinguish between multiple needles hidden in context. This eval is inspired by the MRCR eval first introduced by Gemini ( OpenAI MRCR expands the tasks’s difficulty and provides opensource data for reproducing results. The task is as follows: The model is given a long… See the full description on the dataset page:

Source: Hugging Face Hub (openai/mrcr). Metadata imported from the dataset’s Hub tags.