Senior Data Engineer - Product
仕事内容
As a Senior Data Engineer in this role, you will ensure the DSF platform's stability, scalability, and performance powering real-time risk analytics.
You will work at the intersection of distributed systems, big data engineering, and developer experience, delivering reliable workloads and clean platform components in Java (with Scala or Python as a plus).
Responsibilities include re-architecting and scaling DSF's big data components, analyzing workload patterns, and driving performance, reliability, and cost improvements for Spark jobs on EMR or Kubernetes.
You will operate and evolve Hadoop ecosystem components (HDFS, YARN) and Spark runtimes, and maintain ingestion pipelines (Firehose, Glue → S3).
You will enhance the DSF developer experience via JupyterLabs and the DS API, collaborate cross-functionally, and own services through their full lifecycle with a DevOps mindset.
Requirements include 5+ years in distributed big data, Spark expertise, Java, Hadoop, Linux in cloud environments, ETL/ELT design, and on-call readiness; preferred skills include Spark on EMR/Kubernetes and OSS contributions.
You will work at the intersection of distributed systems, big data engineering, and developer experience, delivering reliable workloads and clean platform components in Java (with Scala or Python as a plus).
Responsibilities include re-architecting and scaling DSF's big data components, analyzing workload patterns, and driving performance, reliability, and cost improvements for Spark jobs on EMR or Kubernetes.
You will operate and evolve Hadoop ecosystem components (HDFS, YARN) and Spark runtimes, and maintain ingestion pipelines (Firehose, Glue → S3).
You will enhance the DSF developer experience via JupyterLabs and the DS API, collaborate cross-functionally, and own services through their full lifecycle with a DevOps mindset.
Requirements include 5+ years in distributed big data, Spark expertise, Java, Hadoop, Linux in cloud environments, ETL/ELT design, and on-call readiness; preferred skills include Spark on EMR/Kubernetes and OSS contributions.
求人の全文を見る
業務内容、求める人物像、スキル、待遇 — 無料アカウントの作成で閲覧できます。
または
すでにアカウントをお持ちですか？
ログイン類似の求人
あなたに合いそうな他の職種。