Data Engineer (1 - 2 years of experience as SDET, Big Data Testing, PySpark)
Is deze vacature iets voor u?
Mijn cv maken Maak uw cv en ontdek uw matchpercentage met deze functie — en met alle andere.
De functie
Develop and maintain PySpark-based test automation scripts for big data applications.
Collaborate with data engineers and developers to understand data processing workflows and requirements.
Design automated tests for data ingestion, processing, and transformations within a Hadoop ecosystem.
Perform data validation, data integrity checks, and performance testing for Spark jobs.
Implement Spark-specific testing using RDDs, DataFrames, Datasets, and Spark SQL.
Set up and maintain CI/CD pipelines for automated testing in a big data environment and ensure high-quality releases.
Collaborate with data engineers and developers to understand data processing workflows and requirements.
Design automated tests for data ingestion, processing, and transformations within a Hadoop ecosystem.
Perform data validation, data integrity checks, and performance testing for Spark jobs.
Implement Spark-specific testing using RDDs, DataFrames, Datasets, and Spark SQL.
Set up and maintain CI/CD pipelines for automated testing in a big data environment and ensure high-quality releases.
Bekijk de volledige vacature
Taken, profiel, vaardigheden en voordelen — maak gratis een account aan.
of
Al een account?
InloggenVergelijkbare vacatures
Andere functies die kunnen passen.
? Supply Chain Data Analyst ? Sr. Data Scientist (Advanced SQL & Python + Spark + ML End-to-End Workflow + Power BI/Tableau) ? Data Science, AI and Automation Specialist ? Senior Software Engineer, Open Banking (AI-first team, Java, AWS, GenAI) ? SW Engineer (1-2 years of experience in Full Stack Development - Java, Angular/React, GenAI) ? Staff Engineer GenAI Developer