Senior Software Engineer (MLOps) – Serving
Is this job for you?
Build my CV Build your CV and see how well you match this role — and every other one.
Track your applications on mobile The free Whileresume app, on iPhone and Android.
The role
Architect and build scalable ML/LLM model-serving systems across data centers with strong SLAs, observability, and reliability.
Design and optimize Ray-based inference infrastructure to handle both low- and high-throughput workloads.
Enable applied scientists to deploy and test models via self-service tools, CI/CD pipelines, and rollback mechanisms.
Implement A/B testing and shadow deployment capabilities to evaluate new model versions in production.
Collaborate with platform teams to improve GPU provisioning, traffic routing, and runtime performance.
Instrument inference workflows with telemetry (latency, token counts, errors) to drive performance and safety analysis.
Design and optimize Ray-based inference infrastructure to handle both low- and high-throughput workloads.
Enable applied scientists to deploy and test models via self-service tools, CI/CD pipelines, and rollback mechanisms.
Implement A/B testing and shadow deployment capabilities to evaluate new model versions in production.
Collaborate with platform teams to improve GPU provisioning, traffic routing, and runtime performance.
Instrument inference workflows with telemetry (latency, token counts, errors) to drive performance and safety analysis.
See the full job post
Responsibilities, requirements, skills and benefits — create your free account.
or
Already have an account?
Log inYou might also like these jobs
No closely matching jobs yet — here are the most recent ones.