Physician (MD/DO) Medical AI Evaluation

About the role

A project-based opportunity for practising US physicians to review clinical vignettes and score the reasoning quality of medical AI benchmark items.

What you will do

• Assess clinical vignettes paired with questions, answers and trap answers across body systems
• Rate benchmark items for accuracy, completeness, answer correctness and trap quality
• Check whether reasoning chains reach the answer from vignette facts alone
• Correct reasoning traces and write brief rationales
• Take part in independent blind reads followed by physician adjudication

What they look for

• MD or DO degree with an active unrestricted US medical licence
• Board certification or completed residency plus recent direct patient care
• Broad multi-system diagnostic and treatment experience in internal, family, emergency or hospital medicine
• Strong clinical reasoning covering differential diagnosis, next steps and evidence-based guidelines
• Written English at C1 level or above