Senior Director, Technical Product Management, AI Quality and Evaluation
- Remote, United States
- Lead
- Posted 9 days ago
or apply from Claude / ChatGPT / Muse
Technical product leadership role on a small incubation team working on AI agents, focused on how quality and evaluation get measured; hands-on and mostly individual contributor.
About the role
The group works like a startup inside a big software vendor, close to AI researchers and engineers and with access to large customers. Strategy, evaluation design and direct customer investigations all sit with this role.
At a glance
- Pay
- Not stated in the posting
- Remote scope
- Remote, United States
- Seniority
- Lead
- Skills
- AI, ML, Coding agents
- Posted
- 7 Nov 2026
- Last checked
- 15 Nov 2026
What you will do
- Set product strategy and a roadmap for AI quality and evaluation that ties customer needs to measurable outcomes
- Make tradeoffs between customer quality work, evaluation capabilities and new features, and keep teams aligned on priorities
- Lead customer investigations and experiments, such as tracing why an agent answered wrong or failed a tool workflow
- Design credible evaluations with researchers and engineers, applying AI and ML basics like train and test separation, overfitting and indepe
- Lead delivery of features that help customers see and improve AI performance, including prototyping with coding agents
What they look for
- Deep AI expertise and a record of turning frontier capabilities into products customers can rely on
- Comfort inspecting traces and data, questioning evaluation results and prototyping with coding agents
- Experience leading through product ownership and cross-functional influence, including coaching junior colleagues
How Keywordise applies for you
Alex Demo
[email protected]
From your CV
Yes, fully remote
- Sent
- 09:41
- Attached
- Your tailored CV
- 1Upload your CVAdd your CV once. It takes about a minute.
- 2Tailored to this jobKeywordise writes a CV and a cover letter for this job from your real experience.
- 3The form fills itselfName, email, experience and screening questions, field by field.
- 4Application sentYou get a receipt and follow it in your dashboard.
Apply with Keywordise AI7-day free trial, no card.
Apply from your AI assistant
Connect Keywordise to your assistant once, then send it the prompt. It applies for you.
Claude
- In Claude, open Settings, then Connectors, and choose Add custom connector.
- Name it Keywordise and paste the connector URL.
- Click Connect, log in to Keywordise and approve. Then send the prompt in a new chat.
https://keywordise.com/mcpDisconnect any time in your Keywordise settings.
ChatGPT
- In ChatGPT, open Settings, then Connectors. On plans that need it, turn on developer mode first.
- Add a new connector with the connector URL and choose OAuth. Leave the client ID and secret empty.
- Log in to Keywordise and approve. Then send the prompt in a new chat.
https://keywordise.com/mcpDisconnect any time in your Keywordise settings.
Muse
- Open Muse (muse.ai or the app) and send it the message below.
- If Muse asks how to sign in, answer OAuth. Open the sign-in link it sends, log in to Keywordise and approve.
- Send the prompt. Muse asks before it sends anything.
Add a custom connector called Keywordise. It is a remote MCP server at https://keywordise.com/mcp and uses OAuth. Give me the sign-in link.Muse is available in the US only for now.
Apply to Keywordise job qnmfnc2vzoQuestions about this job
Is this a remote job, and where can I work from?
Yes, it is listed as remote, aimed at candidates in the United States.
What does this job pay?
The posting does not state a salary, so there is no figure to quote.
What experience is needed?
It is a lead role. The posting asks for: deep AI expertise and a record of turning frontier capabilities into products customers can rely on; comfort inspecting traces and data, questioning evaluation results and prototyping with coding agents; experience leading through product ownership and cross-functional influence, including coaching junior colleagues. Skills it names include AI, ML, Coding agents.
How do I apply with Keywordise AI?
Click Apply with Keywordise AI, create an account and upload your CV. Your CV and cover letter are written for this job from what you have really done, then Keywordise applies for you. Trial: 7 days, no card.
More remote IT and software jobs
PinterestStaff Technical Program Manager, Advanced Technology Group- Remote, United States
- USUSAALead Digital/Technical Product Manager, AI Product Solutions
- Remote, United States
InstacartDirector, AI Solutions Strategy & Planning- Remote, United States
InstacartDirector, AI Solutions Strategy & Planning- CAD 234,000 to 247,000 per year
- Remote, Canada
- BSBentley Systems International LimitedPrincipal Engineer, Global Technical Support Tools and AI Solutions
- Remote, Ireland
GitLabDirector, Data Science- Remote, North America