Physician (MD/DO) - Medical AI Evaluation

About the role

Practicing physicians review clinical vignettes and judge the quality of reasoning behind proposed answers, helping improve medical AI systems. Work is project-based and part-time.

What you will do

• Assess EHR-based clinical vignettes that include a question, a suggested answer and a misleading option
• Rate benchmark items for accuracy, completeness, answer correctness and how easily the answer is given away
• Verify that reasoning follows only from the vignette facts, fix reasoning steps and write brief justifications
• Take part in three separate blind reviews followed by physician adjudication

What they look for

• An MD or DO degree plus an active, unrestricted US medical license
• Board certification or completed residency, with recent hands-on patient care
• Broad experience across multiple body systems in diagnosis and treatment
• Solid clinical reasoning in differential diagnosis, next steps and evidence-based guidelines
• Strong written English at C1 level or above