Physician (MD/DO) Medical AI Evaluation

About the role

Practicing physicians review clinical vignettes and judge the quality of answers produced for a medical AI evaluation program. The work helps improve AI tools used in clinical settings.

What you will do

• Assess clinical vignettes paired with questions, proposed answers and misleading trap answers
• Rate benchmark items for accuracy, completeness, answer correctness and trap quality
• Verify that reasoning chains reach the answer from vignette facts alone and correct them
• Write brief rationales supporting the evaluations
• Take part in independent blind reads followed by physician adjudication

What they look for

• MD or DO degree with an active unrestricted US medical license
• Board certification or completed residency with recent direct patient care
• Broad diagnostic and treatment experience across multiple systems
• Strong clinical reasoning including differential diagnosis and guideline based management
• Written English at C1 level or above

Nice to have

• Prior work in medical AI evaluation, exam question review or clinical annotation and QA