AI Engineer, Product
¿Es esta oferta para usted?
Crear mi CV Cree su CV y descubra su porcentaje de coincidencia con este puesto — y con todos los demás.
Sigue tus candidaturas desde el móvil La app gratuita de Whileresume, en iPhone y Android.
El puesto
Join the product team to design and implement an LLM evaluation framework and end-to-end observability for model releases.
Define and track metrics such as task success, helpfulness, hallucination proxies, safety flags, latency and cost.
Run A/B tests for prompts, models, and system prompts, analyze results, and recommend rollout or rollback decisions.
Set up structured logging, tracing, dashboards and alerts to monitor LLM calls and system health.
Lead model release operations with canary and shadow traffic, SLO-based rollbacks and regression detection.
Create reusable templates and documentation for other teams and collaborate with science to diagnose regressions and post-mortems.
Define and track metrics such as task success, helpfulness, hallucination proxies, safety flags, latency and cost.
Run A/B tests for prompts, models, and system prompts, analyze results, and recommend rollout or rollback decisions.
Set up structured logging, tracing, dashboards and alerts to monitor LLM calls and system health.
Lead model release operations with canary and shadow traffic, SLO-based rollbacks and regression detection.
Create reusable templates and documentation for other teams and collaborate with science to diagnose regressions and post-mortems.
Ver la oferta completa
Funciones, perfil, competencias y ventajas — crea tu cuenta gratis.
o
¿Ya tiene una cuenta?
Iniciar sesiónEstas ofertas podrían interesarte
Ninguna oferta realmente parecida por ahora — estas son las más recientes.