Physician (MD/DO) Medical AI Evaluation

About the role

Practicing physicians review clinical vignettes to test and improve medical AI systems. The work is project-based and part-time, with tasks scored for clinical reasoning quality.

What you will do

• Assess EHR vignettes that pair a question with a proposed answer and a misleading trap answer
• Rate benchmark items on accuracy, completeness, answer correctness and trap quality
• Verify that reasoning chains reach the answer from vignette facts alone and correct traces
• Write brief rationales and take part in blind reads followed by physician adjudication

What they look for

• Medical degree (MD or DO) plus an active unrestricted US license verified with the board
• Board certification or completed residency with recent direct patient care
• Broad diagnostic and treatment experience across multiple systems
• Strong clinical reasoning in differential diagnosis and evidence-based management
• Written English at C1 level or above