Physician (MD/DO) Medical AI Evaluation

About the role

Practicing physicians review clinical vignettes to test and improve medical AI systems. The work is project based and part time, with tasks scored per accepted item.

What you will do

• Assess clinical vignettes that pair a question with a proposed answer and a misleading trap answer
• Rate benchmark items for accuracy, completeness, answer correctness and trap quality
• Verify that reasoning chains reach the answer using only the facts given in the vignette
• Correct reasoning traces and write brief written rationales
• Take part in three separate blind reads followed by physician adjudication

What they look for

• An MD or DO degree plus an active unrestricted US medical license
• Board certification or completed residency with recent direct patient care
• Broad diagnostic and treatment experience across several body systems
• Strong clinical reasoning in differential diagnosis and next-step management
• Written English at C1 level or above