Principal Data Engineer, Data Quality, Datasets Pipelines

Wayve

. Implement and iterate filtering, retention and offload policies across hot, cold, archive and delete storage .

Posted 11/9/2026full-timeSunnyvale • California • United StatesLead💰 $290,000 - $340,000 per yearWebsite

Core Competencies

Role fit

Core Competencies

Use this summary to align your resume positioning with the role.

Demonstrates expertise in building and managing data ingestion and quality pipelines at petabyte scale, with a strong focus on Python and Spark. Capable of implementing effective data filtering, retention, and offload policies while collaborating closely with data infrastructure teams.

Highest-signal resume keywords

Python ProgrammingSpark FrameworkCloud Data EngineeringData Quality ManagementData Ingestion Pipelines

ATS Keywords

Tailor your resume

Applicant Tracking System Keywords

Tip: use these terms in your resume and cover letter to boost ATS matches.

Hard Skills

Data FilteringData Retention PoliciesData Offload StrategiesData MovementDataset Strategy Implementation

Soft Skills

Good JudgementQuick IterationCollaboration

Tools & Technologies

Cloud Platforms

Industry Keywords

Data InfrastructureHigh-Volume IoTRoboticsFleet DataData Lifecycle Policies

Tech Stack

Tools & technologies

CloudIoTPythonSpark

About the role

Key responsibilities & impact

Implement and iterate filtering, retention and offload policies across hot, cold, archive and delete storage
Build data movement and pipelines that the pre-ingestion selection model depends on
Decide where and how selection models run, balancing compute cost against data value
Build early, low-compute decision points that avoid expensive standardisation and enrichment of low-value data
Work with storage and data infrastructure teams and monitor data quality to ensure datasets reflect model-training needs
Turn dataset strategy into working systems for moving, filtering, retaining and offloading driving data at hundreds of petabytes
Deliver measurable storage-cost savings through early, low-compute decisions
Ensure datasets contain useful behavioural and environmental coverage for training
Work as a pair with the dataset strategy lead and closely with the data infrastructure team, shipping practical solutions quickly

Requirements

What you’ll need

Senior data or software engineer who has run data ingestion or data quality pipelines at petabyte scale
Strong in Python and Spark
Cloud data engineering experience
Good judgement on data filtering and quality, including knowing how to check valuable data has not been discarded
Comfortable shipping practical solutions quickly and iterating on live systems
Software engineering background, building systems around models rather than the models themselves
Experience working closely with data infrastructure and storage teams
Experience in AV, robotics, fleet data, high-volume IoT, or retention, offload and data lifecycle policies is a bonus rather than a requirement

Benefits

Comp & perks

Meaningful equity, sharing in the ownership and long term success of Wayve
Relocation support and visa sponsorship where applicable
Hybrid working, core hours and the chance to work hands on in vehicle workshops and labs
Learning and development budgets with support for training, conferences and growth
Comprehensive benefits including health insurance, dental, enhanced maternity and paternity leave, retirement or pension where applicable, access to therapists, wellbeing partner discounts and more
Salaries benchmarked against the market annually