Remote job · Healthcare

Physician (MD/DO) - Medical AI Evaluation

Remote (US)
A company in AI data and evaluation services
Posted 9 Oct 2026

Apply with Keywordise AI

Upload your CV and Keywordise tailors your CV and cover letter to this job and applies for you. 7-day free trial, no card.

Apply with Keywordise AI

About the role

Practicing physicians review clinical EHR vignettes to test and improve medical AI systems. The work is project based and part time, focused on general and internal medicine.

What you will do

Assess EHR vignettes that include a question, a suggested answer and a misleading trap option
Rate benchmark items for accuracy, completeness, answer correctness and trap quality
Judge whether the reasoning chain reaches the answer using only vignette facts
Correct reasoning traces and draft brief written rationales
Take part in three separate blind reads followed by physician adjudication

What they look for

MD or DO degree plus an active unrestricted US medical license
Board certification or completed residency, with recent direct patient care
Experience across several systems in internal, family, emergency or hospital medicine
Solid clinical reasoning in differential diagnosis, next steps and evidence based guidelines
Written English at C1 level or above

How it works

1Upload your CVSign up with Google or your email and add your CV. It takes about a minute.
2We tailor your applicationKeywordise writes a CV and a cover letter for this job from your real experience.
3We apply for youYour application goes to the employer and you follow it in your dashboard.

More remote healthcare jobs

All remote healthcare jobs

Explore more: remote jobs, the Keywordise Job Market Index, the blog.