Software Engineer, AI Code Evaluation

About the role

Engineers assess and grade code produced by AI models on realistic software tasks, and craft test problems with verified answers to sharpen model reasoning.

What you will do

• Judge AI-written code for correctness, maintainability, quality and sound design
• Write demanding programming tasks plus reference solutions and thorough test suites
• Examine model reasoning on architecture, debugging, algorithms and system design
• Spot logic errors, weak edge case handling, slow implementations and design flaws
• Give clear technical feedback and explanations that help models improve
• Keep evaluation criteria consistent so assessments stay accurate and reliable

What they look for

• Three or more years of professional software engineering or data science experience
• At least one year at a top technology, finance or research organisation
• Bachelor's degree in computer science, software engineering, data science or similar from a US, Canadian, Australian, New Zealand or UK univ
• Solid programming and software engineering fundamentals
• Able to check code correctness, find bugs and judge design choices
• Strong analytical skills and excellent written English for technical feedback