Senior Data Engineer

Retail Insight

. Build and maintain scalable data pipelines .

Posted 10/17/2026full-timeChicago • Illinois • United StatesSenior💰 $125,000 per yearWebsite

Core Competencies

Role fit

Core Competencies

Use this summary to align your resume positioning with the role.

Demonstrates expertise in building and maintaining scalable data pipelines, with strong proficiency in Databricks, Apache Spark, and data quality implementation. Collaborates effectively with cross-functional teams to translate business requirements into technical solutions while ensuring data governance and performance optimization.

Highest-signal resume keywords

Data EngineeringDatabricks Pipeline DevelopmentApache SparkPython ProficiencyAzure Data Platforms

ATS Keywords

Tailor your resume

Applicant Tracking System Keywords

Tip: use these terms in your resume and cover letter to boost ATS matches.

Hard Skills

Data Pipeline DevelopmentBatch and Streaming WorkflowsData Quality ChecksETL/ELT PatternsData ModellingSQL ProficiencyDistributed Data ProcessingDatabricks Unity CatalogDelta LakeTerraform

Soft Skills

Strong Communication SkillsCollaborative ApproachOwnership of Quality OutcomesDrive to Learn and Improve

Tools & Technologies

Azure Data FactoryADLS Gen2Azure DevOpsDatabricks WorkflowsKafkaEvent HubsDbt

Industry Keywords

Retail DataCPG Data

Tech Stack

Tools & technologies

ApacheAWSAzureCloudETLGoogle Cloud PlatformKafkaPythonSparkSQLTerraformUnity

About the role

Key responsibilities & impact

Build and maintain scalable data pipelines
Develop and optimise batch and streaming workflows for reliability and performance
Contribute to data architecture design and implementation
Implement data quality checks, monitoring and observability across pipelines
Apply security, access control and governance standards
Collaborate with data scientists and analysts to deliver data products
Translate business requirements into technical solutions with the Lead Data Engineer
Participate in code reviews and contribute to engineering standards
Identify opportunities to optimise cloud costs and platform performance
Contribute to platform documentation, tooling choices and continuous improvement
Work with colleagues and client stakeholders to understand needs and support successful delivery

Requirements

What you’ll need

Solid, demonstrable experience in data engineering, including building and maintaining production-grade Databricks pipelines
Experience with Databricks Unity Catalog and Databricks Asset Bundles
Strong hands-on knowledge of Apache Spark, Delta Lake and Databricks Workflows
Proficiency in Python and SQL
Good understanding of data modelling, ETL/ELT patterns and distributed data processing
Experience with Azure data platforms, including Azure Data Factory, ADLS Gen2 and Azure DevOps, or equivalent services on AWS or GCP
Experience implementing data quality checks and pipeline monitoring
Strong communication skills and ability to work effectively with technical and non-technical colleagues
Collaborative approach, ownership of quality outcomes and drive to learn and improve
A degree in computer science, data engineering or a related field, or equivalent practical experience
Willingness to travel as required
Familiarity with Terraform or Bicep is a bonus
Familiarity with dbt or similar data transformation frameworks is a bonus
Exposure to Kafka or Event Hubs is a bonus
Exposure to retail or CPG data is a bonus
Experience working directly with clients is a bonus

Benefits

Comp & perks

Work from anywhere benefits
Summer hours
Extra days off for life’s big moments, including birthday and wedding
26 days PTO accrued with service
Paid volunteer days
Mental health resources
Access to a top-tier learning platform
Career progression plans
Support to create your ideal home workspace
Comprehensive medical, dental, and vision coverage
6% matched retirement plan
Annual phone reimbursement
Onsite gym access
Customizable benefits via online platform