Physician (MD/DO) Medical AI Evaluation

About the role

Practicing physicians review clinical vignettes to test and improve medical AI systems. The work focuses on diagnosis and treatment reasoning across body systems.

What you will do

• Assess EHR vignettes that pair a question with a proposed answer and a misleading trap answer
• Rate benchmark items for accuracy, completeness, answer correctness, gradeability and trap quality
• Verify that reasoning chains reach conclusions from vignette facts alone and correct traces
• Write brief rationales and take part in blind reads followed by physician adjudication

What they look for

• MD or DO degree plus an active unrestricted US medical license
• Board certification or completed residency with recent direct patient care
• Broad multi-system diagnostic and treatment experience in internal, family, emergency or hospital medicine
• Strong clinical reasoning in differential diagnosis, next steps and evidence-based guidelines
• Strong written English at C1 level or above