AI Engineer, Product
Cette offre est-elle pour vous ?
Créer mon CV Créez votre CV et découvrez votre pourcentage de correspondance avec ce poste — et avec tous les autres.
Suivez vos candidatures sur mobile L'application Whileresume, gratuite, sur iPhone et Android.
Le poste
Contribute to the product team's LLM evaluation framework by designing reference tests, heuristics, and model-graded checks.
Define and monitor metrics such as task success, helpfulness, safety flags, latency, and cost to drive data-driven decisions.
Plan and execute A/B tests for prompts, models, and system prompts, with clear rollout or rollback recommendations.
Implement end-to-end observability for LLM calls, including structured logging, tracing, dashboards, and alerts.
Manage model release processes with canary/shadow traffic, sign-offs, SLO-based rollback criteria, and regression detection.
Improve core behaviors (memory policies, intent classification, follow-ups, routing, and tool-call reliability) and create reusable eval templates for teams.
Define and monitor metrics such as task success, helpfulness, safety flags, latency, and cost to drive data-driven decisions.
Plan and execute A/B tests for prompts, models, and system prompts, with clear rollout or rollback recommendations.
Implement end-to-end observability for LLM calls, including structured logging, tracing, dashboards, and alerts.
Manage model release processes with canary/shadow traffic, sign-offs, SLO-based rollback criteria, and regression detection.
Improve core behaviors (memory policies, intent classification, follow-ups, routing, and tool-call reliability) and create reusable eval templates for teams.
Voir l'offre en entier
Missions, profil recherché, compétences et avantages — en créant votre compte gratuitement.
ou
Déjà un compte ?
Se connecterCes offres pourraient vous intéresser
Aucune offre vraiment proche pour l'instant — voici les plus récentes.