AI Engineer, Product
Is deze vacature iets voor u?
Mijn cv maken Maak uw cv en ontdek uw matchpercentage met deze functie — en met alle andere.
Volg uw sollicitaties op mobiel De gratis Whileresume-app, op iPhone en Android.
De functie
Contribute to the product team's LLM evaluation framework by designing reference tests, heuristics, and model-graded checks.
Define and monitor metrics such as task success, helpfulness, safety flags, latency, and cost to drive data-driven decisions.
Plan and execute A/B tests for prompts, models, and system prompts, with clear rollout or rollback recommendations.
Implement end-to-end observability for LLM calls, including structured logging, tracing, dashboards, and alerts.
Manage model release processes with canary/shadow traffic, sign-offs, SLO-based rollback criteria, and regression detection.
Improve core behaviors (memory policies, intent classification, follow-ups, routing, and tool-call reliability) and create reusable eval templates for teams.
Define and monitor metrics such as task success, helpfulness, safety flags, latency, and cost to drive data-driven decisions.
Plan and execute A/B tests for prompts, models, and system prompts, with clear rollout or rollback recommendations.
Implement end-to-end observability for LLM calls, including structured logging, tracing, dashboards, and alerts.
Manage model release processes with canary/shadow traffic, sign-offs, SLO-based rollback criteria, and regression detection.
Improve core behaviors (memory policies, intent classification, follow-ups, routing, and tool-call reliability) and create reusable eval templates for teams.
Bekijk de volledige vacature
Taken, profiel, vaardigheden en voordelen — maak gratis een account aan.
of
Al een account?
InloggenDeze vacatures zijn misschien iets voor u
Nog geen echt vergelijkbare vacatures — hier zijn de nieuwste.