Lead Data Engineer, Databricks

Booz Allen Hamilton

. Design, build, and maintain robust data pipelines within Databricks .

Posted 10/14/2026full-timeUnited StatesSenior💰 $77,600 - $176,000 per yearWebsite

Core Competencies

Role fit

Core Competencies

Use this summary to align your resume positioning with the role.

Demonstrates expertise in designing and optimizing ETL processes and data pipelines, with a strong focus on data modeling and compliance. Proficient in utilizing Databricks and AWS tools to enhance data processing performance and scalability.

Highest-signal resume keywords

ETL Process DesignDatabricks Pipeline OptimizationPython and PySpark DevelopmentData Modeling ExpertisePublic Trust Eligibility

ATS Keywords

Tailor your resume

Applicant Tracking System Keywords

Tip: use these terms in your resume and cover letter to boost ATS matches.

Hard Skills

ETL ProcessesData PipelinesData ModelingPythonPySparkDatabricksDelta LakeAWS GlueAmazon S3Amazon RDS

Soft Skills

Technical LeadershipCollaborationMentorship

Tools & Technologies

Databricks NotebooksJob Orchestration ToolsUnity Catalog

Industry Keywords

Data SecurityComplianceMedallion ArchitecturePerformance Optimization

Tech Stack

Tools & technologies

AWSETLPySparkPythonUnity

About the role

Key responsibilities & impact

Design, build, and maintain robust data pipelines within Databricks
Create and optimize end-to-end data flows
Handle extract, transform, load (ETL) processes
Design data models and implement schemas and optimized data structures
Ensure data security and compliance
Collaborate with cross-functional teams
Continuously improve data processing performance and scalability
Provide technical leadership and mentorship to analysts, data engineers, developers, and data consumers
Oversee the assessment, design, building, and maintenance of scalable client platforms
Use analytical exploration and data examination to support actionable insights

Requirements

What you’ll need

8+ years of experience designing, developing, deploying, and managing ETL processes and data pipelines utilizing comprehensive logging and robust error handling mechanisms
3+ years of experience building, debugging, and optimizing data pipelines using Databricks notebooks, clusters, and job orchestration tools
Experience with Delta Lake, Databricks Delta, and Unity Catalog
3+ years of experience writing Python and PySpark code for ETL scripts and custom transformations
Experience in data modeling, including dimensional and relational data models within a medallion architecture
Experience using AWS Glue, Amazon S3, and Amazon RDS
Knowledge of ETL performance optimization techniques including partitioning, bucketing, and caching
Ability to design and implement data models from schema definition through data structure optimization
Public Trust
Bachelor’s degree in Computer Science, Computer Engineering, Engineering, Mathematics, Physics, Information Systems, or Information Technology
Applicants selected will be subject to a government investigation and may need to meet eligibility requirements of the U.S. government client
Public Trust determination is required

Benefits

Comp & perks

Health benefits
Life benefits
Disability benefits
Financial benefits
Retirement benefits
Paid leave
Professional development
Tuition assistance
Work-life programs
Dependent care
Recognition awards program for exceptional performance and superior demonstration of values