Senior Machine Learning Engineer, ML Infrastructure (Online)

About the role

Own the design and growth of a platform that serves machine learning models in production at high traffic, with an emphasis on low latency, reliability and safe experimentation.

What you will do

• Build and run large scale online inference systems that serve production models with low latency and high uptime
• Create infrastructure for distributed training workflows and connect ML pipelines to orchestration tools
• Tune model performance through compilation, GPU and CPU efficiency, request scheduling and runtime tuning
• Strengthen monitoring of latency, throughput, errors, cost, saturation and model health
• Work with ML engineers to speed up iteration while keeping production safe, scalable and affordable
• Lead architectural changes that make the platform sturdier, easier to use and cheaper to run

What they look for

• Hands on experience running production online inference systems such as Triton, TorchServe, Ray Serve or TensorFlow Serving
• Track record optimizing inference with dynamic batching, quantization, GPU acceleration, caching or runtime tuning
• Solid background in distributed systems, Kubernetes, autoscaling, service reliability and observability
• Strong Python programming applied to production ML systems and high scale services
• Experience with PyTorch and model deployment workflows covering packaging, validation and serving lifecycle
• Ability to set technical direction and shape architecture across teams without formal authority