Data Engineer (1 - 2 years of experience in Big Data, Spark, Hadoop, Shell scripting)
Cette offre est-elle pour vous ?
Créer mon CV Créez votre CV et découvrez votre pourcentage de correspondance avec ce poste — et avec tous les autres.
Le poste
Design, develop, and maintain automated test frameworks and scripts for large-scale Big Data applications using PySpark.
Collaborate with data engineering, development, and DevOps teams to understand data pipelines, transformation logic, and CI/CD integration.
Build robust tests to validate data ingestion, processing, and transformation in Hadoop and Spark ecosystems; perform data validation and data integrity checks.
Apply deep Spark knowledge (RDDs, DataFrames, Datasets, Spark SQL) to create reusable test automation components and optimize performance.
Leverage Shell scripting and Unix tools for automation, log analysis, environment setup, and troubleshooting.
Maintain documentation, conduct manual and automated testing for critical financial data, and collaborate across time zones to meet SLAs.
Collaborate with data engineering, development, and DevOps teams to understand data pipelines, transformation logic, and CI/CD integration.
Build robust tests to validate data ingestion, processing, and transformation in Hadoop and Spark ecosystems; perform data validation and data integrity checks.
Apply deep Spark knowledge (RDDs, DataFrames, Datasets, Spark SQL) to create reusable test automation components and optimize performance.
Leverage Shell scripting and Unix tools for automation, log analysis, environment setup, and troubleshooting.
Maintain documentation, conduct manual and automated testing for critical financial data, and collaborate across time zones to meet SLAs.
Voir l'offre en entier
Missions, profil recherché, compétences et avantages — en créant votre compte gratuitement.
ou
Déjà un compte ?
Se connecterOffres similaires
D'autres postes qui pourraient vous convenir.
? Supply Chain Data Analyst ? Sr. Data Scientist (Advanced SQL & Python + Spark + ML End-to-End Workflow + Power BI/Tableau) ? Data Science, AI and Automation Specialist ? Data Engineer (1 - 2 years of experience as SDET, Big Data Testing, PySpark) ? Senior Software Engineer, Open Banking (AI-first team, Java, AWS, GenAI) ? SW Engineer (1-2 years of experience in Full Stack Development - Java, Angular/React, GenAI)