Software Engineer, AI Quality & Evaluation
- Remote, United States
- Mid-level
- Posted 4 days ago
or apply from Claude / ChatGPT / Muse
Building backend infrastructure and evaluation systems to test AI applications in telehealth.
About the role
Direct-to-consumer telehealth businesses rely on a platform connecting patient care and pharmacy operations, requiring solid engineering systems to evaluate AI-powered features safely.
At a glance
- Pay
- Not stated in the posting
- Remote scope
- Remote, United States
- Seniority
- Mid-level
- Skills
- APIs, Databases, Python, TypeScript
- Posted
- 12 Nov 2026
- Last checked
- 15 Nov 2026
What you will do
- Design and maintain software infrastructure supporting AI quality and evaluation.
- Build reusable testing tools for large language model integrations.
- Create regression pipelines and result comparison systems.
What they look for
- Strong software engineering fundamentals and experience building backend services.
- Understanding of testing challenges in nondeterministic AI systems.
- Experience working with databases, APIs, and automated testing infrastructure.
- Focus on developer experience and production reliability.
How Keywordise applies for you
Alex Demo
[email protected]
From your CV
Yes, fully remote
- Sent
- 09:41
- Attached
- Your tailored CV
- 1Upload your CVAdd your CV once. It takes about a minute.
- 2Tailored to this jobKeywordise writes a CV and a cover letter for this job from your real experience.
- 3The form fills itselfName, email, experience and screening questions, field by field.
- 4Application sentYou get a receipt and follow it in your dashboard.
Apply with Keywordise AI7-day free trial, no card.
Apply from your AI assistant
Connect Keywordise to your assistant once, then send it the prompt. It applies for you.
Claude
- In Claude, open Settings, then Connectors, and choose Add custom connector.
- Name it Keywordise and paste the connector URL.
- Click Connect, log in to Keywordise and approve. Then send the prompt in a new chat.
https://keywordise.com/mcpDisconnect any time in your Keywordise settings.
ChatGPT
- In ChatGPT, open Settings, then Connectors. On plans that need it, turn on developer mode first.
- Add a new connector with the connector URL and choose OAuth. Leave the client ID and secret empty.
- Log in to Keywordise and approve. Then send the prompt in a new chat.
https://keywordise.com/mcpDisconnect any time in your Keywordise settings.
Muse
- Open Muse (muse.ai or the app) and send it the message below.
- If Muse asks how to sign in, answer OAuth. Open the sign-in link it sends, log in to Keywordise and approve.
- Send the prompt. Muse asks before it sends anything.
Add a custom connector called Keywordise. It is a remote MCP server at https://keywordise.com/mcp and uses OAuth. Give me the sign-in link.Muse is available in the US only for now.
Apply to Keywordise job iex6bkf2zwQuestions about this job
Is this job fully remote, and from where?
Yes. The posting lists Software Engineer, AI Quality & Evaluation as a remote role for candidates in the United States.
What does this job pay?
The posting does not state a salary, so there is no figure to quote.
What experience is needed?
It is a mid-level role. The posting asks for: strong software engineering fundamentals and experience building backend services; understanding of testing challenges in nondeterministic AI systems; experience working with databases, APIs, and automated testing infrastructure. Skills it names include APIs, Databases, Python, TypeScript.
How do I apply with Keywordise AI?
Press the Apply with Keywordise AI button and add your CV. Keywordise tailors your CV and cover letter to this posting and sends the application for you, with a free 7-day trial and no card.
More remote IT and software jobs
- NNorthbeamSoftware Engineer, Python
- USD 140,000 to 155,000 per year
- Remote, Canada
- RRyderApplication Development and Support Engineer II
- Remote, United States
- A large insurance and financial services companySenior Data Scientist/AI Application Solutions Engineer
- Remote, United States
- A customer experience software companyForward Deployed AI Engineer
- Remote, United States
- A data quality platform companyAI Content Engineer
- Remote, United Kingdom
- A remote analytics infrastructure companyPlatform Engineer
- Remote, United States