(USA) Principal, Data Scientist - Gen AI Vision
Cette offre est-elle pour vous ?
Créer mon CV Créez votre CV et découvrez votre pourcentage de correspondance avec ce poste — et avec tous les autres.
Suivez vos candidatures sur mobile L'application Whileresume, gratuite, sur iPhone et Android.
Le poste
Own the evaluation architecture for AI agents, designing how success is measured across non-deterministic outputs.
Lead the brain that critiques agents with a mix of LLM-based judging, human benchmarks, and automated pipelines.
Develop evaluation pipelines, calibrate prompts, and distill smaller judge models balancing accuracy, latency, and cost.
Curate large-scale conversation data to build reliable Golden Set datasets and standardized ground truth for subjective tasks.
Collaborate with engineering to integrate quality signals into CI/CD pipelines for automated testing and monitoring.
Drive insights to identify systemic weaknesses, mentor data scientists, and advance best practices in AI evaluation.
Lead the brain that critiques agents with a mix of LLM-based judging, human benchmarks, and automated pipelines.
Develop evaluation pipelines, calibrate prompts, and distill smaller judge models balancing accuracy, latency, and cost.
Curate large-scale conversation data to build reliable Golden Set datasets and standardized ground truth for subjective tasks.
Collaborate with engineering to integrate quality signals into CI/CD pipelines for automated testing and monitoring.
Drive insights to identify systemic weaknesses, mentor data scientists, and advance best practices in AI evaluation.
Voir l'offre en entier
Missions, profil recherché, compétences et avantages — en créant votre compte gratuitement.
ou
Déjà un compte ?
Se connecterCes offres pourraient vous intéresser
Aucune offre vraiment proche pour l'instant — voici les plus récentes.