Senior or Staff Data Scientist, Applied AI

About the role

You will set the quality and reliability bar for an AI assistant inside a trading platform, building evaluation systems, benchmarks and monitoring that keep its behaviour safe and predictable.

What you will do

• Design evaluation systems measuring correctness, safety, latency and regression risk
• Build and maintain benchmark sets, scenario suites and adversarial test cases
• Create automated quality gates that block releases when key metrics drop
• Define safe tool and action contracts with engineering and product
• Run model improvement loops covering data collection, error analysis and tuning
• Operate monitoring and incident response for the AI, including telemetry and alerting

What they look for

• Eight or more years shipping production software with strong skills in any language
• Solid computer science fundamentals, testing methods and systems design
• Experience building evaluation frameworks and benchmark suites for complex systems
• Experience running model improvement cycles with measurable benchmark gains
• Comfort working across the stack, debugging failures and instrumenting services