Senior Software Engineer (MLOps) – Annotation & Evaluation
Is deze vacature iets voor u?
Mijn cv maken Maak uw cv en ontdek uw matchpercentage met deze functie — en met alle andere.
Volg uw sollicitaties op mobiel De gratis Whileresume-app, op iPhone en Android.
De functie
Design and build systems for automated evaluation of AI models, including LLMs and agents, using production-like telemetry and realistic scenarios.
Lead the development of benchmark suites, evaluation pipelines, and model comparison tools with integrated trust & safety metrics.
Build and maintain integrations with labeling systems (e.g., Label Studio) and coordinate with external/internal annotation workflows.
Collaborate with Applied AI and Bits AI teams to enable fast iteration, reproducible experiments, and interpretable evaluations.
Develop data pipelines that feed metrics, results, and alerts into our observability stack to monitor model behavior at scale.
Promote safe deployment practices via bias checks, hallucination detection, and human-in-the-loop review.
Lead the development of benchmark suites, evaluation pipelines, and model comparison tools with integrated trust & safety metrics.
Build and maintain integrations with labeling systems (e.g., Label Studio) and coordinate with external/internal annotation workflows.
Collaborate with Applied AI and Bits AI teams to enable fast iteration, reproducible experiments, and interpretable evaluations.
Develop data pipelines that feed metrics, results, and alerts into our observability stack to monitor model behavior at scale.
Promote safe deployment practices via bias checks, hallucination detection, and human-in-the-loop review.
Bekijk de volledige vacature
Taken, profiel, vaardigheden en voordelen — maak gratis een account aan.
of
Al een account?
InloggenDeze vacatures zijn misschien iets voor u
Nog geen echt vergelijkbare vacatures — hier zijn de nieuwste.