24 MagHiring

Senior Software Engineer, LLM Evaluation

  • Remote, United States
  • Contract
  • Senior
  • Posted 9 days ago
Apply with Keywordise AI

or apply from Claude / ChatGPT / Muse

In one line

Part-time remote contract for experienced engineers who review AI-written code, curate examples and build checks for coding benchmarks. Open to people in the US, Canada or Western Europe.

About the role

Code curation and evaluation sit at the centre: judging model output against professional standards, writing structured rationales, and helping research teams see where models fail across the development lifecycle.

At a glance

Pay
Not stated in the posting
Remote scope
Remote, United States
Seniority
Senior
Employment
Contract
Skills
Python, JavaScript, Java, Rust, Go
Posted
20 Oct 2026
Last checked
27 Oct 2026

What you will do

  • Curate code samples used for model training and benchmarking
  • Judge AI-generated code for correctness, efficiency and reliability
  • Write structured rationales that back each evaluation decision
  • Build agents and automated checks that verify software solutions
  • Review model behaviour across design, API, launch and maintenance stages

What they look for

  • Three or more years of professional software engineering
  • Full-stack skills and experience with production-scale systems
  • Command of Python, JavaScript, Java, C++, Rust or similar
  • Clear written and spoken communication, including structured reasoning

Nice to have

  • ReactJS, C or Go experience
  • Familiarity with API design and operational monitoring

How Keywordise applies for you

  1. 1Upload your CVAdd your CV once. It takes about a minute.
  2. 2Tailored to this jobKeywordise writes a CV and a cover letter for this job from your real experience.
  3. 3The form fills itselfName, email, experience and screening questions, field by field.
  4. 4Application sentYou get a receipt and follow it in your dashboard.

Apply with Keywordise AI7-day free trial, no card.

Apply from your AI assistant

Connect Keywordise to your assistant once, then send it the prompt. It applies for you.

ClaudeConnectorChatGPTConnectorMuseby Meta

Claude

  1. In Claude, open Settings, then Connectors, and choose Add custom connector.
  2. Name it Keywordise and paste the connector URL.
  3. Click Connect, log in to Keywordise and approve. Then send the prompt in a new chat.
Connector URL
https://keywordise.com/mcp

Disconnect any time in your Keywordise settings.

ChatGPT

  1. In ChatGPT, open Settings, then Connectors. On plans that need it, turn on developer mode first.
  2. Add a new connector with the connector URL and choose OAuth. Leave the client ID and secret empty.
  3. Log in to Keywordise and approve. Then send the prompt in a new chat.
Connector URL
https://keywordise.com/mcp

Disconnect any time in your Keywordise settings.

Muse

  1. Open Muse (muse.ai or the app) and send it the message below.
  2. If Muse asks how to sign in, answer OAuth. Open the sign-in link it sends, log in to Keywordise and approve.
  3. Send the prompt. Muse asks before it sends anything.
Message for Muse
Add a custom connector called Keywordise. It is a remote MCP server at https://keywordise.com/mcp and uses OAuth. Give me the sign-in link.

Muse is available in the US only for now.

Prompt
Apply to Keywordise job 3s7uavqq26
How to connect

Questions about this job

Can I do this job remotely, and from which countries?

Yes. The posting lists Senior Software Engineer, LLM Evaluation as a remote role for candidates in the United States.

What does this job pay?

The posting does not state a salary, so there is no figure to quote.

What experience is needed?

It is a senior role. The posting asks for: three or more years of professional software engineering; full-stack skills and experience with production-scale systems; command of Python, JavaScript, Java, C++, Rust or similar. Skills it names include Python, JavaScript, Java, Rust.

How do I apply with Keywordise AI?

Select Apply with Keywordise AI, sign up and upload your CV. Keywordise writes a CV and a cover letter for this job from your real experience and applies for you. The 7-day free trial needs no card.

More remote IT and software jobs

All remote IT and software jobs
24 MagRemote, United States
Apply with Keywordise AI