Accessing the Benchmark Dataset

Option 1. Download exams in csv format

download zip (85 KB)

Option 2. HuggingFace

from datasets import load_dataset

ds = load_dataset("nejm-ai-qa/exams")

ds contains the 5 exams:

DatasetDict({
    general_surgery: Dataset({
        features: ['question', 'answer'],
        num_rows: 141
    })
    internal_medicine: Dataset({
        features: ['question', 'answer'],
        num_rows: 126
    })
    psychiatry: Dataset({
        features: ['question', 'answer'],
        num_rows: 150
    })
    pediatrics: Dataset({
        features: ['question', 'answer'],
        num_rows: 99
    })
    obgyn: Dataset({
        features: ['question', 'answer'],
        num_rows: 139
    })
})

Fetch the first question-answer pair:

first_exam = ds['general_surgery']
first_qa = first_exam[0]

Print the question:

print(first_qa['question'])

A 23-year-old is undergoing surgery for an incarcerated femoral hernia. As part of the surgery, a cut was made of the ligament named after Cooper, and started bleeding. For which artery should super-selective angiography be performed?
A. Obturator artery.
B. Superior epigastric artery.
C. InferiorEpigastricArtery.
D. Femoral Artery.

Print the answer:

print(first_qa['answer'])

A