Physician (MD/DO) Medical AI Evaluation

About the role

Practicing physicians review clinical vignettes to test and improve medical AI systems. The work is project-based and part-time, aimed at raising the quality of diagnostic and treatment reasoning in AI tools.

What you will do

• Assess clinical vignettes that pair a case with a question, a suggested answer and a misleading option
• Rate benchmark items for accuracy, completeness, answer correctness and how well the trap works
• Verify that reasoning reaches the answer from case facts alone, fix reasoning steps and write brief justifications
• Take part in three separate blind reviews followed by physician adjudication

What they look for

• MD or DO degree plus an active unrestricted US medical license
• Board certification or completed residency, with recent direct patient care
• Broad diagnostic and treatment experience across body systems
• Solid clinical reasoning in differential diagnosis, next steps and evidence-based guidelines
• Strong written English at C1 level or above