Physician (MD/DO), Medical AI Evaluation

About the role

Practicing physicians review clinical EHR vignettes and score the quality of medical AI benchmark items, helping improve the reasoning of tools clinicians may later use.

What you will do

• Evaluate clinical vignettes paired with questions, proposed answers and trap answers across diagnosis and treatment tasks
• Score items for vignette accuracy and completeness, whether the answer is given away, answer correctness and gradeability, and trap quality
• Check that reasoning chains reach the answer from vignette facts alone and correct reasoning traces
• Write short rationales and take part in three independent blind reads followed by physician adjudication

What they look for

• MD or DO with an active, unrestricted US medical license verified by the issuing board
• Board certification or completed residency, with recent direct patient care, attending level preferred
• Broad multi-system diagnostic and treatment experience in internal, family, emergency or hospital medicine
• Strong clinical reasoning in differential diagnosis, next-step management and evidence-based guidelines
• Written English at C1 or above