Physician (MD/DO), Medical AI Evaluation

About the role

Project-based work for practising physicians who assess clinical vignettes and reasoning quality to help improve medical AI systems.

What you will do

• Evaluate clinical EHR vignettes paired with questions, proposed answers and trap answers across diagnosis and treatment
• Score benchmark items for vignette accuracy, answer correctness, gradeability and trap quality
• Check whether reasoning chains reach the answer from vignette facts alone and write short rationales
• Work through three independent blind reads followed by physician adjudication

What they look for

• MD or DO degree with an active, unrestricted US medical licence
• Board certification or completed residency, with recent direct patient care
• Broad multi-system diagnostic and treatment experience in internal, family, emergency or hospital medicine
• Strong clinical reasoning including differential diagnosis and evidence-based management
• Strong written English at C1 level or above