Senior Data Engineer

Paul Hastings

. Design, build and manage scalable data pipelines using Azure Databricks .

Posted 9/23/2026full-timeUnited StatesSenior💰 $125,800 - $194,750 per yearWebsite

Core Competencies

Role fit

Core Competencies

Use this summary to align your resume positioning with the role.

Demonstrates advanced proficiency in Azure Databricks and Azure DevOps for building scalable data pipelines and implementing CI/CD practices. Strong expertise in SQL, PySpark, and data modeling, with a focus on performance optimization and data security best practices.

Highest-signal resume keywords

Azure DatabricksAzure DevOpsSQLPySparkData Engineering

ATS Keywords

Tailor your resume

Applicant Tracking System Keywords

Tip: use these terms in your resume and cover letter to boost ATS matches.

Hard Skills

Data Pipeline DevelopmentELT FrameworksCI/CD Pipeline ManagementData ModelingVersion Control with GitData Security Best PracticesPerformance TuningAgile MethodologiesData TransformationStructured and Semi-Structured Data

Soft Skills

CollaborationMentoringProblem Solving

Tools & Technologies

Azure MonitorLog AnalyticsPower BIInfrastructure-as-CodeAzure BoardsJIRAMicrosoft Purview

Certifications & Qualifications

Bachelor’s Degree in Computer ScienceEngineering or Related Field

Industry Keywords

Professional ServicesLegalFinancialData GovernanceData Cataloging

Tech Stack

Tools & technologies

AzureCloudPySparkPythonSQL

About the role

Key responsibilities & impact

Design, build and manage scalable data pipelines using Azure Databricks
Develop ELT frameworks for ingesting and transforming structured and semi-structured data from on-premises and cloud sources, including cloud-based APIs
Implement DevOps pipelines for code versioning, testing, deployment and monitoring using Azure DevOps
Ensure pipeline performance, reliability and observability using Azure Monitor and Log Analytics
Collaborate with BI teams to support Power BI datasets and optimize data model performance
Automate environment setup, configuration and permissions using infrastructure-as-code where possible
Write modular, testable code in SQL, Python or PySpark and manage version control with Git
Support production operations, root cause analysis and performance tuning
Understand data sources and data relationship models
Maintain technical documentation for pipelines, data mappings and deployment processes
Participate in Agile sprints, retrospectives and code reviews
Mentor junior data engineers and promote best practices in CI/CD, version control and release management

Requirements

What you’ll need

Bachelor’s degree in Computer Science, Engineering or related field (required)
7+ years of data engineering or BI development experience
3+ years building data solutions in Microsoft Azure Cloud
2+ years using Azure DevOps for automated builds, releases and pipeline orchestration
Advanced proficiency in Azure Databricks
Experience building and managing CI/CD pipelines in Azure DevOps for data solutions
Strong SQL and ELT experience; familiarity with schema design (e.g., star, snowflake, denormalized)
Hands-on expertise with PySpark and Databricks notebooks, including Delta Lake workflows
Familiarity with Git, branching strategies and repository management within Azure Repos
Understanding of data security and privacy best practices in Azure (RBAC, managed identities, encryption)
Experience with Power BI dataflows, datasets and refresh configuration (preferred)
Familiarity with data cataloging and governance tools (e.g., Microsoft Purview) a plus
Experience with Agile tools such as Azure Boards or JIRA
Proven experience working with structured and semi-structured data (Parquet, JSON, Avro)
Experience in professional services, legal, or financial environments preferred

Benefits

Comp & perks

Generous total compensation package with opportunity to earn bonuses based on individual contribution and firm profitability
Medical, Dental, Vision, Life/AD&D, Long Term Care, and Short- and Long-Term Disability
Flexible Spending Accounts and Health Savings Account
Healthcare Concierge and Advocacy
Lifestyle Spending Account
Voluntary 401(k) Plan and Profit Sharing
10 Paid Holidays per year and a generous PTO Program
Family Support including Paid Parental Leave, Fertility Benefits, Breast Milk Shipping, Back-up Child Care, Elder Care, and Tutoring
Wellbeing programs (Employee Assistance Program, Relationship Support, Mental Health, Menopause and Midlife Health and Well-Being Events)
Retirement Plan Consulting
Anniversary Bonus Program
Professional Development Programs
Transportation and Commuter Benefits
International Travel Insurance
Critical Illness, Hospital Indemnity and Accident Insurance
Auto/Home/Pet Insurance
Prepaid Legal Insurance
Employee Discounts