Senior Data Engineer - Product
仕事内容
Re-architect and scale the DSF-based big data processing components powering production risk analytics.
Analyze workload patterns across Spark jobs, notebooks, and DS API usage to drive performance, reliability, and cost improvements.
Ensure stability of Spark jobs running on EMR or Kubernetes and operate Hadoop ecosystem components (HDFS, YARN) and Spark runtimes.
Maintain and evolve ingestion pipelines between Runtime and DSF (Firehose, Glue → S3) and improve Spark runtimes.
Enhance developer and power-user experience across JupyterLabs and the DS API.
Own services throughout their lifecycle following DevOps practices, collaborating with data scientists and platform teams to deliver a reliable, scalable data platform.
Analyze workload patterns across Spark jobs, notebooks, and DS API usage to drive performance, reliability, and cost improvements.
Ensure stability of Spark jobs running on EMR or Kubernetes and operate Hadoop ecosystem components (HDFS, YARN) and Spark runtimes.
Maintain and evolve ingestion pipelines between Runtime and DSF (Firehose, Glue → S3) and improve Spark runtimes.
Enhance developer and power-user experience across JupyterLabs and the DS API.
Own services throughout their lifecycle following DevOps practices, collaborating with data scientists and platform teams to deliver a reliable, scalable data platform.
求人の全文を見る
業務内容、求める人物像、スキル、待遇 — 無料アカウントの作成で閲覧できます。
または
すでにアカウントをお持ちですか？
ログイン類似の求人
あなたに合いそうな他の職種。