Physician (MD/DO) Medical AI Evaluation

About the role

Practicing physicians review clinical vignettes and judge the quality of answers and reasoning produced for a medical AI evaluation program, working on a part-time project basis.

What you will do

• Score items for vignette accuracy and completeness and for whether the answer is given away
• Judge answer correctness, gradeability and the quality of the trap
• Verify that reasoning traces reach the answer from vignette facts alone and correct them
• Write brief rationales and take part in blind reads followed by physician adjudication

What they look for

• MD or DO degree plus an active unrestricted US medical license
• Board certification or completed residency with recent direct patient care
• Broad multi-system diagnostic and treatment experience in internal, family, emergency or hospital medicine
• Strong clinical reasoning covering differential diagnosis, next-step management and evidence-based guidelines
• Written English at C1 or above

Nice to have

• Medical annotation or quality assurance experience