Physician (MD/DO) Medical AI Evaluation

About the role

Practicing physicians review clinical vignettes and reasoning for a medical AI evaluation program, mainly in general and internal medicine. Work is part-time and project-based.

What you will do

• Assess EHR vignettes paired with questions, proposed answers and misleading trap answers across body systems
• Rate benchmark items for vignette accuracy, completeness, answer correctness and trap quality
• Verify that reasoning chains reach the answer from vignette facts alone and correct reasoning traces
• Write brief rationales and take part in blind reads followed by physician adjudication

What they look for

• MD or DO degree with an active unrestricted US medical license
• Board certification or completed residency plus recent direct patient care, attending level preferred
• Broad multi-system diagnostic and treatment experience in internal, family, emergency or hospital medicine
• Solid clinical reasoning covering differential diagnosis, next steps and evidence-based guidelines
• Written English at C1 level or above