Senior Data Engineer

People Data Labs

. Build infrastructure for ingestion, transformation and loading of increasing data volumes from varied sources using Spark, SQL, AWS and Databricks .

Posted 10/27/2026full-timeRemote • United StatesSenior💰 $200,000 - $220,000 per yearWebsite

Core Competencies

Role fit

Core Competencies

Use this summary to align your resume positioning with the role.

Demonstrates expertise in building scalable data processing systems using Apache Spark, SQL, and AWS, while effectively collaborating with teams to solve complex data engineering challenges. Proficient in developing CI/CD pipelines and anomaly detection systems to enhance data quality and integrity.

Highest-signal resume keywords

Apache Spark ExpertisePython ProgrammingData Pipeline DevelopmentAWS Cloud ServicesData Warehousing Experience

ATS Keywords

Tailor your resume

Applicant Tracking System Keywords

Tip: use these terms in your resume and cover letter to boost ATS matches.

Hard Skills

Apache SparkPythonSQLData Pipeline DevelopmentCI/CD PipelinesAnomaly DetectionData Design DocumentationMedallion ArchitectureData Lakehouse PatternsIncremental Updating

Soft Skills

Strong Written CommunicationCollaborationProblem-SolvingProject ScopingAutonomy

Tools & Technologies

DatabricksAirflowDbtDAGsterDelta Live TablesSnowflakeRedshiftBigQueryParquetORC

Industry Keywords

Data EngineeringData ScienceCloud ComputingData WarehousingData Storage Formats

Tech Stack

Tools & technologies

AirflowAmazon RedshiftApacheAWSAzureBigQueryCloudGoogle Cloud PlatformJavaPySparkPythonScalaSnowflakeSparkSQL

About the role

Key responsibilities & impact

Build infrastructure for ingestion, transformation and loading of increasing data volumes from varied sources using Spark, SQL, AWS and Databricks
Build an organic entity resolution framework to merge hundreds of billions of individual entities into clean, consumable datasets
Develop CI/CD pipelines and anomaly detection systems to continuously improve production data quality
Develop solutions to undefined data engineering and data science problems
Collaborate with teammates, managers and stakeholders while scoping projects and communicating progress and blockers

Requirements

What you’ll need

5-7+ years of industry experience with clear examples of strategic technical problem-solving and implementation
Strong software development fundamentals
Experience with Python
Expertise with Apache Spark (Java, Scala, and/or Python-based)
Experience with SQL
Experience building scalable data processing systems from the ground up
Experience using developer-oriented data pipeline and workflow orchestration, such as Airflow, dbt, dagster or similar
Knowledge of modern data design and storage patterns, including incremental updating, partitioning and segmentation, rebuilds and backfills
Experience working in Databricks, including Delta Live Tables and data lakehouse patterns
Experience with cloud computing services, such as AWS, GCP, Azure or similar
Experience with data warehousing, such as Databricks, Snowflake, Redshift, BigQuery or similar
Understanding of modern data storage formats and tools, such as Parquet, ORC, Avro and Delta Lake
Ability to balance high ownership and autonomy with collaboration
Ability to work effectively remotely and proactively manage blockers, reach out, ask questions and participate in team activities
Strong written communication skills on Slack/Chat and in documents
Experience writing data design documents, including pipeline design, dataflow and schema design
Ability to scope and break down projects and communicate progress and blockers effectively
Professional experience with Spark or PySpark
Experience with medallion architecture
Authorization to work in the United States
Degree in a quantitative discipline is listed as a nice to have, not a requirement

Benefits

Comp & perks

Stock
Competitive Salaries
Unlimited paid time off
Medical, dental, & vision insurance
Health, fitness, and office stipends
The permanent ability to work wherever and however you want