Physician (MD/DO) Medical AI Evaluation

About the role

Practicing physicians review clinical vignettes to test and improve medical AI systems. The work is project based and part time, with tasks submitted during active phases.

What you will do

• Assess EHR case vignettes that pair a question with a proposed answer and a misleading trap answer
• Rate benchmark items for clinical reasoning, including accuracy, completeness and whether the answer is revealed
• Judge answer correctness and gradeability along with the quality of the trap option
• Verify that reasoning chains reach the conclusion from case facts alone and correct faulty traces
• Write brief rationales and take part in blind reads followed by physician adjudication

What they look for

• Medical degree (MD or DO) plus an active unrestricted US license verified by the issuing board
• Board certification or completed residency, with recent direct patient care, ideally at attending level
• Broad diagnostic and treatment experience across systems in internal, family, emergency or hospital medicine
• Solid clinical reasoning in differential diagnosis, next steps and evidence based guidelines
• Strong written English at C1 level or above