Senior Research Engineer, LLM Evaluation and Behavioral Analysis
Is this job for you?
Build my CV Build your CV and see how well you match this role — and every other one.
The role
Lead the design and iteration of evaluation frameworks that measure LLM behavior across instruction following, reasoning, tool use, and multi-turn interactions.
Develop specialized suites for function calling, including argument correctness, schema adherence, tool selection, multi-function planning, and error recovery.
Build agentic workflow evaluations that test task decomposition, multi-step planning, self-correction, and autonomous tool-use sequences.
Create CI/CD pipelines for A/B comparisons, regression detection, behavioral drift monitoring, and adversarial probing.
Curate high-quality evaluation datasets with nuanced and challenging cases across domains to guide model improvements.
Collaborate with researchers, engineers, and product teams to diagnose failures, triage regressions, and visualize behavior changes across releases.
Develop specialized suites for function calling, including argument correctness, schema adherence, tool selection, multi-function planning, and error recovery.
Build agentic workflow evaluations that test task decomposition, multi-step planning, self-correction, and autonomous tool-use sequences.
Create CI/CD pipelines for A/B comparisons, regression detection, behavioral drift monitoring, and adversarial probing.
Curate high-quality evaluation datasets with nuanced and challenging cases across domains to guide model improvements.
Collaborate with researchers, engineers, and product teams to diagnose failures, triage regressions, and visualize behavior changes across releases.
See the full job post
Responsibilities, requirements, skills and benefits — create your free account.
or
Already have an account?
Log inSimilar openings
Other roles that could suit you.