Physician (MD/DO), Medical AI Evaluation

About the role

Practicing physicians review clinical vignettes and answers to test medical AI systems. Work is project-based and part-time, with tasks scored on clinical reasoning quality.

What you will do

• Evaluate vignettes paired with questions, answers and trap answers across body systems
• Score reasoning quality, including vignette accuracy and answer correctness
• Check whether reasoning chains reach the answer from the facts given
• Correct reasoning traces and write short rationales
• Take part in blind reads followed by physician adjudication

What they look for

• MD or DO degree and an active unrestricted US license
• Board certification or completed residency with recent patient care
• Broad diagnostic and treatment experience across systems
• Strong clinical reasoning and use of evidence-based guidelines
• Strong written English at C1 level or above

Nice to have

• Prior work in medical AI evaluation or exam-question review
• Experience in medical annotation or quality assurance