24 MagHiring

Senior Software Engineer, AI Evaluation / Coding Agents

  • USD 100 to 150 per hour
  • Remote, United States
  • Freelance
  • Senior
  • Posted 9 days ago
Apply with Keywordise AI

or apply from Claude / ChatGPT / Muse

In one line

Freelance role reviewing code from AI coding agents and building evaluation rubrics; remote for North America, 20 to 40 hours per week.

About the role

Repository-level analysis of AI-generated code and rubric design to judge whether coding agents produce correct, maintainable software. The focus is evaluation rather than building production applications.

At a glance

Pay
USD 100 to 150 per hour
Remote scope
Remote, United States
Seniority
Senior
Employment
Freelance
Skills
Python, TypeScript, JavaScript, Go, Java, GitHub, CI/CD, RLHF
Posted
20 Oct 2026
Last checked
27 Oct 2026

What you will do

  • Review code produced by AI coding agents for correctness and maintainability
  • Design and refine technical evaluation rubrics
  • Analyse code changes within substantial real-world repositories
  • Build and maintain pipelines for data generation and evaluation
  • Summarise findings in written reports for research and engineering teams

What they look for

  • 5+ years of hands-on software engineering experience
  • Experience working in substantial real-world codebases
  • Strong code-review and technical-analysis skills
  • Familiarity with CI/CD and production engineering processes

Nice to have

  • Familiarity with open-source development
  • LLM evaluation or coding-agent experience

How Keywordise applies for you

  1. 1Upload your CVAdd your CV once. It takes about a minute.
  2. 2Tailored to this jobKeywordise writes a CV and a cover letter for this job from your real experience.
  3. 3The form fills itselfName, email, experience and screening questions, field by field.
  4. 4Application sentYou get a receipt and follow it in your dashboard.

Apply with Keywordise AI7-day free trial, no card.

Apply from your AI assistant

Connect Keywordise to your assistant once, then send it the prompt. It applies for you.

ClaudeConnectorChatGPTConnectorMuseby Meta

Claude

  1. In Claude, open Settings, then Connectors, and choose Add custom connector.
  2. Name it Keywordise and paste the connector URL.
  3. Click Connect, log in to Keywordise and approve. Then send the prompt in a new chat.
Connector URL
https://keywordise.com/mcp

Disconnect any time in your Keywordise settings.

ChatGPT

  1. In ChatGPT, open Settings, then Connectors. On plans that need it, turn on developer mode first.
  2. Add a new connector with the connector URL and choose OAuth. Leave the client ID and secret empty.
  3. Log in to Keywordise and approve. Then send the prompt in a new chat.
Connector URL
https://keywordise.com/mcp

Disconnect any time in your Keywordise settings.

Muse

  1. Open Muse (muse.ai or the app) and send it the message below.
  2. If Muse asks how to sign in, answer OAuth. Open the sign-in link it sends, log in to Keywordise and approve.
  3. Send the prompt. Muse asks before it sends anything.
Message for Muse
Add a custom connector called Keywordise. It is a remote MCP server at https://keywordise.com/mcp and uses OAuth. Give me the sign-in link.

Muse is available in the US only for now.

Prompt
Apply to Keywordise job 5gkc7md2h2
How to connect

Questions about this job

Is this a remote job, and where can I work from?

The posting lists it as remote for people in the United States.

What does this job pay?

The posting states USD 100 to 150 per hour.

What experience is needed?

It is a senior role. The posting asks for: 5+ years of hands-on software engineering experience; experience working in substantial real-world codebases; strong code-review and technical-analysis skills. Skills it names include Python, TypeScript, JavaScript, Go.

How do I apply with Keywordise AI?

Click Apply with Keywordise AI, create an account and upload your CV. Your CV and cover letter are written for this job from what you have really done, then Keywordise applies for you. Trial: 7 days, no card.

More remote IT and software jobs

All remote IT and software jobs
24 MagRemote, United States
Apply with Keywordise AI