AI Engineer, Product
在手机上跟进您的申请 Whileresume 免费应用，支持 iPhone 和 Android。
职位介绍
Join the product team to design and implement an LLM evaluation framework and end-to-end observability for model releases.
Define and track metrics such as task success, helpfulness, hallucination proxies, safety flags, latency and cost.
Run A/B tests for prompts, models, and system prompts, analyze results, and recommend rollout or rollback decisions.
Set up structured logging, tracing, dashboards and alerts to monitor LLM calls and system health.
Lead model release operations with canary and shadow traffic, SLO-based rollbacks and regression detection.
Create reusable templates and documentation for other teams and collaborate with science to diagnose regressions and post-mortems.
Define and track metrics such as task success, helpfulness, hallucination proxies, safety flags, latency and cost.
Run A/B tests for prompts, models, and system prompts, analyze results, and recommend rollout or rollback decisions.
Set up structured logging, tracing, dashboards and alerts to monitor LLM calls and system health.
Lead model release operations with canary and shadow traffic, SLO-based rollbacks and regression detection.
Create reusable templates and documentation for other teams and collaborate with science to diagnose regressions and post-mortems.
查看完整职位
工作职责、任职要求、技能与福利 — 免费创建账号即可查看。
或
已有账户？
登录您可能也感兴趣的职位
暂无高度相似的职位 — 以下是最新职位。