Physician (MD/DO), Medical AI Evaluation

About the role

Practising physicians review clinical EHR vignettes and score the quality of medical AI benchmark items. Work is project-based and part-time, done in blind reads with later adjudication.

What you will do

• Assess EHR vignettes paired with questions, answers and trap answers
• Score clinical reasoning quality, including vignette accuracy and answer correctness
• Check whether reasoning chains reach the answer from vignette facts alone
• Write short rationales and correct reasoning traces
• Take part in independent blind reads followed by physician adjudication

What they look for

• MD or DO with an active, unrestricted US medical licence
• Board certification or completed residency, plus recent direct patient care
• Broad multi-system diagnostic and treatment experience
• Strong clinical reasoning and knowledge of evidence-based guidelines
• Written English at C1 level or above

Nice to have

• Experience with medical annotation or quality assurance