Physician (MD/DO) Medical AI Evaluation

About the role

Practicing physicians review clinical vignettes to test and improve medical AI systems. Work is project based and done in blind reads with physician adjudication.

What you will do

• Assess EHR vignettes that pair a question with a proposed answer and a misleading trap answer
• Rate benchmark items for accuracy, completeness, answer correctness and trap quality
• Verify that reasoning chains reach the answer using only the vignette facts
• Correct reasoning traces and write brief written rationales
• Take part in three independent blind reads before physician adjudication

What they look for

• Medical degree (MD or DO) plus an active unrestricted US license
• Board certification or completed residency with recent direct patient care
• Broad diagnostic and treatment experience across body systems
• Strong clinical reasoning including differential diagnosis and next-step management
• Written English at C1 level or above

Nice to have

• Experience with medical annotation or quality assurance