Physician Medical AI Evaluation (MD/DO)

About the role

Practicing physicians review clinical vignettes to test and improve medical AI systems. Work is project-based and part-time.

What you will do

• Rate benchmark items for accuracy, completeness and how easily the answer is revealed
• Check that reasoning reaches the answer from the vignette facts alone
• Correct reasoning traces and write brief rationales
• Take part in three independent blind reads followed by physician adjudication

What they look for

• Board certification or completed residency with recent direct patient care
• Broad diagnostic and treatment experience across body systems
• Strong clinical reasoning including differential diagnosis and next-step management
• Written English at C1 level or above

Nice to have

• Experience with medical annotation or quality assurance