Manager – Data Engineering

Marsh McLennan

. Design and implement scalable data pipelines using Databricks Lakehouse architecture .

Posted 10/5/2026full-timePune • IndiaMid-LevelSeniorWebsite

Core Competencies

Role fit

Core Competencies

Use this summary to align your resume positioning with the role.

Demonstrates expertise in designing and implementing scalable data pipelines using Databricks and Spark, with a strong focus on Medallion Architecture and Delta Lake solutions. Proficient in data quality practices, governance, and optimizing large-scale data processing for enterprise analytics.

Highest-signal resume keywords

DatabricksSparkMedallion ArchitectureDelta LakePySpark

ATS Keywords

Tailor your resume

Applicant Tracking System Keywords

Tip: use these terms in your resume and cover letter to boost ATS matches.

Hard Skills

Data Pipeline DevelopmentSQL Query OptimizationData TransformationData Quality ChecksIncremental ProcessingData AnalysisPerformance TuningData ModelingData GovernanceData Lineage

Tools & Technologies

Databricks LakehouseDelta LakeInformaticaAWS

Industry Keywords

Data EngineeringCloud ComputingEnterprise AnalyticsLarge-Scale Distributed Data ProcessingData Observability

Tech Stack

Tools & technologies

AWSCloudInformaticaPySparkSparkSQL

About the role

Key responsibilities & impact

Design and implement scalable data pipelines using Databricks Lakehouse architecture
Build and maintain data products aligned with Medallion Architecture
Develop ingestion, transformation, and consumption pipelines
Design business-ready Gold layer datasets for reporting, analytics, and downstream consumption
Build and manage Delta Lake tables and data assets
Implement incremental processing using Merge, Upsert, and CDC patterns
Optimize Delta tables through partitioning, compaction, and performance tuning
Analyze existing data flows and SQLs and translate legacy workflows into optimized PySpark pipelines
Document transformation logic and migration approaches
Build scalable PySpark applications on Databricks
Develop reusable frameworks and components
Tune Spark jobs and analyze execution plans
Implement data quality checks, reconciliation processes, and validation frameworks
Write complex SQL queries, stored procedures, and optimized data transformations
Perform data analysis, profiling, and troubleshooting using SQL
Optimize queries and data models for large-scale datasets
Support data lineage, metadata management, governance initiatives, CI/CD, and deployment automation
Collaborate with data architects, business stakeholders, analysts, and platform teams

Requirements

What you’ll need

Strong hands-on experience with Databricks and Spark-based data engineering
Experience implementing Medallion Architecture (Bronze, Silver, Gold layers)
Experience designing and managing Delta Lake solutions
Good understanding of Lakehouse Architecture principles
Experience developing scalable data pipelines for enterprise analytics platforms
Knowledge of data quality, lineage, governance, and observability practices
Prior experience with Informatica is advantageous
Cloud experience, preferably AWS
Hands-on experience with Databricks, PySpark, SQL, Delta Lake, Medallion Architecture, data modeling, and large-scale distributed data processing

Benefits

Comp & perks

Professional development opportunities
Interesting work and supportive leaders
Vibrant and inclusive culture
Flexible work environment
Range of career opportunities
Benefits and rewards to enhance well-being
Flexibility of working remotely
Collaboration, connections and professional development benefits of working together in the office