Lead Data Engineer - (Datawarehouse) - Apache Nifi, Python, PySpark, Hadoop, Cloudera platforms, and Airflow
Is deze vacature iets voor u?
Mijn cv maken Maak uw cv en ontdek uw matchpercentage met deze functie — en met alle andere.
Volg uw sollicitaties op mobiel De gratis Whileresume-app, op iPhone en Android.
De functie
Lead the design and delivery of secure, scalable data pipelines and data warehouse solutions in a Big Data environment using PySpark (Python/Scala) on Hadoop or object storage.
Build and optimize end-to-end data pipelines and petabyte-scale data architectures, leveraging Nifi pipelines when available and Airflow for workflow orchestration.
Work across on-premises and cloud platforms (AWS, Azure, Databricks) and manage Hive external tables, partitions, and multiple file formats.
Collaborate in Agile/Scrum teams, create well-defined user stories, perform SQL tuning (Oracle, Netezza), and implement code quality via Git and CI/CD (Jenkins).
Drive automation and efficiency in data ingestion, movement and access workflows; diagnose production issues and implement robust mitigations.
Demonstrate initiative and strong communication while juggling multiple projects in distributed teams and upholding security standards.
Build and optimize end-to-end data pipelines and petabyte-scale data architectures, leveraging Nifi pipelines when available and Airflow for workflow orchestration.
Work across on-premises and cloud platforms (AWS, Azure, Databricks) and manage Hive external tables, partitions, and multiple file formats.
Collaborate in Agile/Scrum teams, create well-defined user stories, perform SQL tuning (Oracle, Netezza), and implement code quality via Git and CI/CD (Jenkins).
Drive automation and efficiency in data ingestion, movement and access workflows; diagnose production issues and implement robust mitigations.
Demonstrate initiative and strong communication while juggling multiple projects in distributed teams and upholding security standards.
Bekijk de volledige vacature
Taken, profiel, vaardigheden en voordelen — maak gratis een account aan.
of
Al een account?
InloggenDeze vacatures zijn misschien iets voor u
Nog geen echt vergelijkbare vacatures — hier zijn de nieuwste.