Physician (MD/DO) - Medical AI Evaluation

About the role

Practicing physicians review clinical EHR vignettes to test and improve medical AI systems. Work is project-based and part-time, with a focus on general and internal medicine.

What you will do

• Assess EHR vignettes that include a question, a proposed answer and a misleading trap answer
• Rate benchmark items for vignette accuracy, completeness, answer correctness and trap quality
• Verify that reasoning chains reach the answer using only vignette facts, and fix traces
• Write brief rationales and take part in three blind reads followed by physician adjudication

What they look for

• MD or DO degree plus an active unrestricted US medical license
• Board certification or completed residency, with recent direct patient care
• Broad diagnostic and treatment experience across multiple systems
• Strong clinical reasoning in differential diagnosis and next-step management
• Written English at C1 level or above