Physician (MD/DO) Medical AI Evaluation

About the role

Practicing physicians review clinical vignettes to test and improve medical AI systems. The work is project based and done part time.

What you will do

• Assess EHR vignettes that pair a case with a question, an answer and a misleading option
• Rate benchmark items for accuracy, completeness, answerability and trap quality
• Verify that reasoning chains reach the answer using only the vignette facts
• Correct reasoning traces and write brief justifications
• Take part in three separate blind reviews followed by physician adjudication

What they look for

• MD or DO degree with an active unrestricted US medical license
• Board certification or completed residency plus recent direct patient care
• Broad experience across multiple systems in internal, family, emergency or hospital medicine
• Solid clinical reasoning including differential diagnosis and next-step management
• Strong written English at C1 level or above