Lead Data Engineer - (Datawarehouse) - Apache Nifi, Python, PySpark, Hadoop, Cloudera platforms, and Airflow
Cette offre est-elle pour vous ?
Créer mon CV Créez votre CV et découvrez votre pourcentage de correspondance avec ce poste — et avec tous les autres.
Suivez vos candidatures sur mobile L'application Whileresume, gratuite, sur iPhone et Android.
Le poste
Lead the design and delivery of secure, scalable data pipelines and data warehouse solutions in a Big Data environment using PySpark (Python/Scala) on Hadoop or object storage.
Build and optimize end-to-end data pipelines and petabyte-scale data architectures, leveraging Nifi pipelines when available and Airflow for workflow orchestration.
Work across on-premises and cloud platforms (AWS, Azure, Databricks) and manage Hive external tables, partitions, and multiple file formats.
Collaborate in Agile/Scrum teams, create well-defined user stories, perform SQL tuning (Oracle, Netezza), and implement code quality via Git and CI/CD (Jenkins).
Drive automation and efficiency in data ingestion, movement and access workflows; diagnose production issues and implement robust mitigations.
Demonstrate initiative and strong communication while juggling multiple projects in distributed teams and upholding security standards.
Build and optimize end-to-end data pipelines and petabyte-scale data architectures, leveraging Nifi pipelines when available and Airflow for workflow orchestration.
Work across on-premises and cloud platforms (AWS, Azure, Databricks) and manage Hive external tables, partitions, and multiple file formats.
Collaborate in Agile/Scrum teams, create well-defined user stories, perform SQL tuning (Oracle, Netezza), and implement code quality via Git and CI/CD (Jenkins).
Drive automation and efficiency in data ingestion, movement and access workflows; diagnose production issues and implement robust mitigations.
Demonstrate initiative and strong communication while juggling multiple projects in distributed teams and upholding security standards.
Voir l'offre en entier
Missions, profil recherché, compétences et avantages — en créant votre compte gratuitement.
ou
Déjà un compte ?
Se connecterCes offres pourraient vous intéresser
Aucune offre vraiment proche pour l'instant — voici les plus récentes.
? Lead/ Staff Data Scientist (Bangkok based, relocation provided) ? Staff Research Scientist - Quantum Computing ? Senior Software Engineer - Machine Learning ? Lead/Staff Data Scientist (Bangkok-based, relocation provided) ? Staff Research Scientist - Generative AI Security (GATE) - Visa Research ? Senior Software Engineer - Machine Learning