Physician (MD/DO) Medical AI Evaluation

About the role

Practicing physicians review clinical vignettes to test and improve medical AI systems. Work is project based and part time.

What you will do

• Assess EHR case vignettes that include a question, a suggested answer and a misleading option
• Rate benchmark items for clinical reasoning quality, including accuracy, completeness and trap quality
• Verify that reasoning steps lead to the answer using only the vignette facts, and write brief rationales
• Correct reasoning traces and take part in three blind reads followed by physician adjudication

What they look for

• MD or DO degree with an active unrestricted US medical license
• Board certification or completed residency, plus recent direct patient care
• Broad diagnostic and treatment experience across multiple systems
• Strong clinical reasoning in differential diagnosis, next steps and evidence based guidelines
• Strong written English at C1 level or above