(USA) Principal, Data Scientist - Gen AI Vision
Ist diese Stelle etwas für Sie?
Lebenslauf erstellen Erstellen Sie Ihren Lebenslauf und entdecken Sie Ihre Übereinstimmung mit dieser Stelle — und mit allen anderen.
Die Stelle
Own the evaluation architecture for AI agents, designing how success is measured across non-deterministic outputs.
Lead the brain that critiques agents with a mix of LLM-based judging, human benchmarks, and automated pipelines.
Develop evaluation pipelines, calibrate prompts, and distill smaller judge models balancing accuracy, latency, and cost.
Curate large-scale conversation data to build reliable Golden Set datasets and standardized ground truth for subjective tasks.
Collaborate with engineering to integrate quality signals into CI/CD pipelines for automated testing and monitoring.
Drive insights to identify systemic weaknesses, mentor data scientists, and advance best practices in AI evaluation.
Lead the brain that critiques agents with a mix of LLM-based judging, human benchmarks, and automated pipelines.
Develop evaluation pipelines, calibrate prompts, and distill smaller judge models balancing accuracy, latency, and cost.
Curate large-scale conversation data to build reliable Golden Set datasets and standardized ground truth for subjective tasks.
Collaborate with engineering to integrate quality signals into CI/CD pipelines for automated testing and monitoring.
Drive insights to identify systemic weaknesses, mentor data scientists, and advance best practices in AI evaluation.
Die vollständige Anzeige sehen
Aufgaben, Anforderungen, Kompetenzen und Vorteile — mit Ihrem kostenlosen Konto.
oder
Bereits ein Konto?
AnmeldenÄhnliche Stellen
Weitere Positionen, die passen könnten.