Physician (MD/DO) - Medical AI Evaluation

About the role

Practicing physicians review clinical vignettes to assess and improve medical AI systems. The work is project-based and part-time, focused on diagnosis and treatment reasoning.

What you will do

• Assess clinical vignettes paired with questions, answers and misleading trap answers
• Rate benchmark items for accuracy, completeness, answer quality and trap design
• Verify that reasoning chains reach conclusions from vignette facts alone
• Correct reasoning traces and write brief rationales
• Take part in blind reads followed by physician adjudication

What they look for

• MD or DO degree with an active unrestricted US medical license
• Board certification or completed residency with recent direct patient care
• Broad multi-system diagnostic and treatment experience
• Strong clinical reasoning across differential diagnosis and management
• Strong written English at C1 level or above

Nice to have

• Prior work in medical AI evaluation, clinical content review or medical annotation