Physician (MD/DO) Medical AI Evaluation

About the role

Evaluate clinical vignettes and reasoning for a medical AI evaluation program, helping improve the AI systems used in clinical work.

What you will do

• Assess clinical vignettes paired with questions, proposed answers and trap answers across diagnosis and treatment
• Rate benchmark items for accuracy, completeness, answer correctness and trap quality
• Check whether reasoning chains reach answers from vignette facts alone and write brief rationales
• Take part in three independent blind reads followed by physician adjudication

What they look for

• MD or DO degree plus an active, unrestricted US medical license
• Board certification or completed residency, with recent direct patient care
• Broad diagnostic and treatment experience across multiple systems
• Strong clinical reasoning, including differential diagnosis and evidence-based guidelines
• Strong written English at C1 level or above