Data Engineer (1 - 2 years of experience in Big Data, Spark, Hadoop, Shell scripting)
¿Es esta oferta para usted?
Crear mi CV Cree su CV y descubra su porcentaje de coincidencia con este puesto — y con todos los demás.
El puesto
Design, develop, and maintain automated test frameworks and scripts for large-scale Big Data applications using PySpark.
Collaborate with data engineering, development, and DevOps teams to understand data pipelines, transformation logic, and CI/CD integration.
Build robust tests to validate data ingestion, processing, and transformation in Hadoop and Spark ecosystems; perform data validation and data integrity checks.
Apply deep Spark knowledge (RDDs, DataFrames, Datasets, Spark SQL) to create reusable test automation components and optimize performance.
Leverage Shell scripting and Unix tools for automation, log analysis, environment setup, and troubleshooting.
Maintain documentation, conduct manual and automated testing for critical financial data, and collaborate across time zones to meet SLAs.
Collaborate with data engineering, development, and DevOps teams to understand data pipelines, transformation logic, and CI/CD integration.
Build robust tests to validate data ingestion, processing, and transformation in Hadoop and Spark ecosystems; perform data validation and data integrity checks.
Apply deep Spark knowledge (RDDs, DataFrames, Datasets, Spark SQL) to create reusable test automation components and optimize performance.
Leverage Shell scripting and Unix tools for automation, log analysis, environment setup, and troubleshooting.
Maintain documentation, conduct manual and automated testing for critical financial data, and collaborate across time zones to meet SLAs.
Ver la oferta completa
Funciones, perfil, competencias y ventajas — crea tu cuenta gratis.
o
¿Ya tiene una cuenta?
Iniciar sesiónOfertas similares
Otros puestos que podrían encajar.
? Supply Chain Data Analyst ? Sr. Data Scientist (Advanced SQL & Python + Spark + ML End-to-End Workflow + Power BI/Tableau) ? Data Science, AI and Automation Specialist ? Data Engineer (1 - 2 years of experience as SDET, Big Data Testing, PySpark) ? Senior Software Engineer, Open Banking (AI-first team, Java, AWS, GenAI) ? SW Engineer (1-2 years of experience in Full Stack Development - Java, Angular/React, GenAI)