Physician (MD/DO) - Medical AI Evaluation

About the role

Project based work for practising physicians who review clinical vignettes and score the reasoning quality of medical AI benchmark items. Contributors work in blind reads with physician adjudication.

What you will do

• Evaluate clinical EHR vignettes with a question, a proposed answer and a trap answer across diagnosis and treatment tasks
• Score clinical reasoning quality, including vignette accuracy, completeness and whether the answer is given away
• Check answer correctness and gradeability and judge trap quality
• Verify that reasoning chains reach the answer from vignette facts alone and correct reasoning traces

What they look for

• Board certification or completed residency with recent direct patient care, attending level preferred
• Broad multi system diagnostic and treatment experience in internal, family, emergency or hospital medicine
• Strong clinical reasoning covering differential diagnosis, next step management and evidence based guidelines
• Strong written English at C1 or above