Program Manager, Human Evaluations (Recommendations)
- Remote, United States
- Full-time
- Lead
- Posted 8 days ago
or apply from Claude / ChatGPT / Muse
Program manager running human evaluation programs that judge how good recommendations are on a streaming homepage; works closely with product and machine learning teams.
About the role
Human evaluation work for recommendation quality: building rubrics and workflows, managing raters and vendors, and keeping delivery on schedule. The role pairs evaluation design with hands-on execution.
At a glance
- Pay
- Not stated in the posting
- Remote scope
- Remote, United States
- Seniority
- Lead
- Employment
- Full-time
- Skills
- Human evaluation, Rubrics, Vendor management, Success metrics, Personalization
- Posted
- 21 Oct 2026
- Last checked
- 27 Oct 2026
What you will do
- Design evaluation frameworks and rubrics for relevance, repetition and diversity
- Run evaluations end to end, from intake and scoping through delivery
- Onboard and calibrate raters and manage vendor capacity
- Translate recommendation strategy into evaluation designs and success metrics
- Work with product, research, data science, UX and engineering teams
What they look for
- Experience running human evaluation or data operations programs
- Able to bring structure to unclear problems and keep projects on track
- Comfort working with product managers, data scientists and ML engineers
- Fluency in film and TV content and a feel for which titles fit together
- Skill at defining milestones and steering cross-functional teams
How Keywordise applies for you
Alex Demo
[email protected]
From your CV
Yes, fully remote
- Sent
- 09:41
- Attached
- Your tailored CV
- 1Upload your CVAdd your CV once. It takes about a minute.
- 2Tailored to this jobKeywordise writes a CV and a cover letter for this job from your real experience.
- 3The form fills itselfName, email, experience and screening questions, field by field.
- 4Application sentYou get a receipt and follow it in your dashboard.
Apply with Keywordise AI7-day free trial, no card.
Apply from your AI assistant
Connect Keywordise to your assistant once, then send it the prompt. It applies for you.
Claude
- In Claude, open Settings, then Connectors, and choose Add custom connector.
- Name it Keywordise and paste the connector URL.
- Click Connect, log in to Keywordise and approve. Then send the prompt in a new chat.
https://keywordise.com/mcpDisconnect any time in your Keywordise settings.
ChatGPT
- In ChatGPT, open Settings, then Connectors. On plans that need it, turn on developer mode first.
- Add a new connector with the connector URL and choose OAuth. Leave the client ID and secret empty.
- Log in to Keywordise and approve. Then send the prompt in a new chat.
https://keywordise.com/mcpDisconnect any time in your Keywordise settings.
Muse
- Open Muse (muse.ai or the app) and send it the message below.
- If Muse asks how to sign in, answer OAuth. Open the sign-in link it sends, log in to Keywordise and approve.
- Send the prompt. Muse asks before it sends anything.
Add a custom connector called Keywordise. It is a remote MCP server at https://keywordise.com/mcp and uses OAuth. Give me the sign-in link.Muse is available in the US only for now.
Apply to Keywordise job wjhpsnyxscQuestions about this job
Is this a remote job, and where can I work from?
Yes. The posting lists Program Manager, Human Evaluations (Recommendations) as a remote role for candidates in the United States.
What does this job pay?
The posting does not state a salary, so there is no figure to quote.
What experience is needed?
It is a lead role. The posting asks for: experience running human evaluation or data operations programs; able to bring structure to unclear problems and keep projects on track; comfort working with product managers, data scientists and ML engineers. Skills it names include Human evaluation, Rubrics, Vendor management, Success metrics.
How do I apply with Keywordise AI?
Select Apply with Keywordise AI, sign up and upload your CV. Keywordise writes a CV and a cover letter for this job from your real experience and applies for you. The 7-day free trial needs no card.
More remote management jobs
- IImmaticsPrincipal Manager, Clinical Operations Lead
- USD 165,000 to 180,000 per year
- Remote, United States
- A financial services and technology companyProduct Manager
- Remote, United States
- A direct-to-consumer vehicle manufacturerPrincipal Product Manager, AI Personalization and Discovery
- USD 156,000 to 260,000 per year
- Remote, United States
- HRHard Rock DigitalCRM Casino Manager
- Remote, worldwide
- CCareSourceProduct Manager, Data Migration
- Remote, worldwide
- RCRocket CompaniesSenior Product Manager, Growth and Conversion
- USD 145,700 to 197,100 per year
- Remote, United States