Physician (MD/DO) - Medical AI Evaluation

About the role

Practicing physicians review clinical vignettes to help test and improve medical AI systems. The work is project-based and part-time, with tasks completed remotely.

What you will do

• Assess clinical EHR vignettes that include a question, a suggested answer and a misleading trap answer
• Score benchmark items for vignette accuracy, completeness, answer correctness, gradeability and trap quality
• Judge whether the reasoning chain reaches the answer using only vignette facts, fix reasoning traces and write brief rationales
• Take part in three separate blind reviews followed by physician adjudication

What they look for

• An MD or DO and a current unrestricted US medical licence verified by the issuing board
• Board certification or completed residency, with recent direct patient care, attending level preferred
• Broad diagnostic and treatment experience across body systems in internal, family, emergency or hospital medicine
• Solid clinical reasoning in differential diagnosis, next steps and evidence-based guidelines
• Written English at C1 level or above