Senior Data Engineer - Product
Is deze vacature iets voor u?
Mijn cv maken Maak uw cv en ontdek uw matchpercentage met deze functie — en met alle andere.
De functie
As a Senior Data Engineer in this role, you will ensure the DSF platform's stability, scalability, and performance powering real-time risk analytics.
You will work at the intersection of distributed systems, big data engineering, and developer experience, delivering reliable workloads and clean platform components in Java (with Scala or Python as a plus).
Responsibilities include re-architecting and scaling DSF's big data components, analyzing workload patterns, and driving performance, reliability, and cost improvements for Spark jobs on EMR or Kubernetes.
You will operate and evolve Hadoop ecosystem components (HDFS, YARN) and Spark runtimes, and maintain ingestion pipelines (Firehose, Glue → S3).
You will enhance the DSF developer experience via JupyterLabs and the DS API, collaborate cross-functionally, and own services through their full lifecycle with a DevOps mindset.
Requirements include 5+ years in distributed big data, Spark expertise, Java, Hadoop, Linux in cloud environments, ETL/ELT design, and on-call readiness; preferred skills include Spark on EMR/Kubernetes and OSS contributions.
You will work at the intersection of distributed systems, big data engineering, and developer experience, delivering reliable workloads and clean platform components in Java (with Scala or Python as a plus).
Responsibilities include re-architecting and scaling DSF's big data components, analyzing workload patterns, and driving performance, reliability, and cost improvements for Spark jobs on EMR or Kubernetes.
You will operate and evolve Hadoop ecosystem components (HDFS, YARN) and Spark runtimes, and maintain ingestion pipelines (Firehose, Glue → S3).
You will enhance the DSF developer experience via JupyterLabs and the DS API, collaborate cross-functionally, and own services through their full lifecycle with a DevOps mindset.
Requirements include 5+ years in distributed big data, Spark expertise, Java, Hadoop, Linux in cloud environments, ETL/ELT design, and on-call readiness; preferred skills include Spark on EMR/Kubernetes and OSS contributions.
Bekijk de volledige vacature
Taken, profiel, vaardigheden en voordelen — maak gratis een account aan.
of
Al een account?
InloggenVergelijkbare vacatures
Andere functies die kunnen passen.