Senior Data Engineer - Product
Is this job for you?
Build my CV Build your CV and see how well you match this role — and every other one.
Track your applications on mobile The free Whileresume app, on iPhone and Android.
The role
As a Senior Data Engineer in this role, you will ensure the DSF platform's stability, scalability, and performance powering real-time risk analytics.
You will work at the intersection of distributed systems, big data engineering, and developer experience, delivering reliable workloads and clean platform components in Java (with Scala or Python as a plus).
Responsibilities include re-architecting and scaling DSF's big data components, analyzing workload patterns, and driving performance, reliability, and cost improvements for Spark jobs on EMR or Kubernetes.
You will operate and evolve Hadoop ecosystem components (HDFS, YARN) and Spark runtimes, and maintain ingestion pipelines (Firehose, Glue → S3).
You will enhance the DSF developer experience via JupyterLabs and the DS API, collaborate cross-functionally, and own services through their full lifecycle with a DevOps mindset.
Requirements include 5+ years in distributed big data, Spark expertise, Java, Hadoop, Linux in cloud environments, ETL/ELT design, and on-call readiness; preferred skills include Spark on EMR/Kubernetes and OSS contributions.
You will work at the intersection of distributed systems, big data engineering, and developer experience, delivering reliable workloads and clean platform components in Java (with Scala or Python as a plus).
Responsibilities include re-architecting and scaling DSF's big data components, analyzing workload patterns, and driving performance, reliability, and cost improvements for Spark jobs on EMR or Kubernetes.
You will operate and evolve Hadoop ecosystem components (HDFS, YARN) and Spark runtimes, and maintain ingestion pipelines (Firehose, Glue → S3).
You will enhance the DSF developer experience via JupyterLabs and the DS API, collaborate cross-functionally, and own services through their full lifecycle with a DevOps mindset.
Requirements include 5+ years in distributed big data, Spark expertise, Java, Hadoop, Linux in cloud environments, ETL/ELT design, and on-call readiness; preferred skills include Spark on EMR/Kubernetes and OSS contributions.
See the full job post
Responsibilities, requirements, skills and benefits — create your free account.
or
Already have an account?
Log inYou might also like these jobs
No closely matching jobs yet — here are the most recent ones.