Data Engineer (1 - 2 years of experience in Big Data, Spark, Hadoop, Shell scripting)
Is deze vacature iets voor u?
Mijn cv maken Maak uw cv en ontdek uw matchpercentage met deze functie — en met alle andere.
De functie
Design, develop, and maintain automated test frameworks and scripts for large-scale Big Data applications using PySpark.
Collaborate with data engineering, development, and DevOps teams to understand data pipelines, transformation logic, and CI/CD integration.
Build robust tests to validate data ingestion, processing, and transformation in Hadoop and Spark ecosystems; perform data validation and data integrity checks.
Apply deep Spark knowledge (RDDs, DataFrames, Datasets, Spark SQL) to create reusable test automation components and optimize performance.
Leverage Shell scripting and Unix tools for automation, log analysis, environment setup, and troubleshooting.
Maintain documentation, conduct manual and automated testing for critical financial data, and collaborate across time zones to meet SLAs.
Collaborate with data engineering, development, and DevOps teams to understand data pipelines, transformation logic, and CI/CD integration.
Build robust tests to validate data ingestion, processing, and transformation in Hadoop and Spark ecosystems; perform data validation and data integrity checks.
Apply deep Spark knowledge (RDDs, DataFrames, Datasets, Spark SQL) to create reusable test automation components and optimize performance.
Leverage Shell scripting and Unix tools for automation, log analysis, environment setup, and troubleshooting.
Maintain documentation, conduct manual and automated testing for critical financial data, and collaborate across time zones to meet SLAs.
Bekijk de volledige vacature
Taken, profiel, vaardigheden en voordelen — maak gratis een account aan.
of
Al een account?
InloggenVergelijkbare vacatures
Andere functies die kunnen passen.
? Supply Chain Data Analyst ? Sr. Data Scientist (Advanced SQL & Python + Spark + ML End-to-End Workflow + Power BI/Tableau) ? Data Science, AI and Automation Specialist ? Data Engineer (1 - 2 years of experience as SDET, Big Data Testing, PySpark) ? Senior Software Engineer, Open Banking (AI-first team, Java, AWS, GenAI) ? SW Engineer (1-2 years of experience in Full Stack Development - Java, Angular/React, GenAI)