AI Engineer, Product
応募状況をスマホで確認 Whileresume の無料アプリ（iPhone・Android）。
仕事内容
Join the product team to design and implement an LLM evaluation framework and end-to-end observability for model releases.
Define and track metrics such as task success, helpfulness, hallucination proxies, safety flags, latency and cost.
Run A/B tests for prompts, models, and system prompts, analyze results, and recommend rollout or rollback decisions.
Set up structured logging, tracing, dashboards and alerts to monitor LLM calls and system health.
Lead model release operations with canary and shadow traffic, SLO-based rollbacks and regression detection.
Create reusable templates and documentation for other teams and collaborate with science to diagnose regressions and post-mortems.
Define and track metrics such as task success, helpfulness, hallucination proxies, safety flags, latency and cost.
Run A/B tests for prompts, models, and system prompts, analyze results, and recommend rollout or rollback decisions.
Set up structured logging, tracing, dashboards and alerts to monitor LLM calls and system health.
Lead model release operations with canary and shadow traffic, SLO-based rollbacks and regression detection.
Create reusable templates and documentation for other teams and collaborate with science to diagnose regressions and post-mortems.
求人の全文を見る
業務内容、求める人物像、スキル、待遇 — 無料アカウントの作成で閲覧できます。
または
すでにアカウントをお持ちですか？
ログインこちらの求人もおすすめです
近い求人はまだありません — 最新の求人をご紹介します。