Data Engineer

EXL

. Build and maintain pipelines to land six in-scope sources into the Fabric Bronze/raw layer .

Posted 10/9/2026full-timeGurugram • IndiaMid-LevelSeniorWebsite

Core Competencies

Role fit

Core Competencies

Use this summary to align your resume positioning with the role.

Demonstrates expertise in building and maintaining data pipelines, implementing data quality checks, and optimizing performance in data engineering environments. Proficient in PySpark, SQL, and Microsoft Fabric tools for effective data ingestion and processing.

Highest-signal resume keywords

Data Engineering ExperiencePySpark ProficiencySQL ExpertiseMicrosoft Fabric KnowledgeDelta Lake Familiarity

ATS Keywords

Tailor your resume

Applicant Tracking System Keywords

Tip: use these terms in your resume and cover letter to boost ATS matches.

Hard Skills

Data Pipeline DevelopmentIncremental Load ImplementationChange Data Capture (CDC)Data Quality ChecksSpark NotebooksBatch IngestionSchema-on-ReadName NormalizationAddress ParsingAttribute Standardization

Soft Skills

Problem SolvingCollaborationAttention to Detail

Tools & Technologies

Microsoft FabricDelta LakeSparkData FactoryOneLake

Certifications & Qualifications

Bachelor's Degree

Industry Keywords

Medallion ArchitectureBronze/Silver/Gold LayeringData Quality ValidationPipeline MonitoringMetadata Capture

Tech Stack

Tools & technologies

PySparkPythonSparkSQL

About the role

Key responsibilities & impact

Build and maintain pipelines to land six in-scope sources into the Fabric Bronze/raw layer
Implement Fabric Mirroring for supported structured sources and establish change-data-capture patterns
Implement watermark and incremental-load logic where mirroring is unavailable
Maintain one append-only Delta table per source, retaining evidence records and full source provenance
Implement name normalization, address parsing, and attribute-standardization logic in Spark notebooks
Support identifier-spine construction
Implement data quality checks, validation rules, threshold alerts, and exception handling
Support reconciliation against source data
Schedule, monitor, and troubleshoot pipeline runs
Investigate failures and performance issues
Optimize Spark jobs, Delta file sizes, partitioning, and pipeline efficiency to manage Fabric capacity consumption
Produce and maintain source-to-target mappings, transformation-logic documentation, and lineage records
Cross-train on Splink tuning and candidate-pair generation
Support the Sr. Data Engineer (Lead) and assist the VectorDB Engineer with document/attribute preparation in Phase 2

Requirements

What you’ll need

Bachelor's Degree listed as the study level
4+ years hands-on data engineering experience with strong PySpark and SQL
Production experience building ingestion pipelines from multiple heterogeneous sources
Working knowledge of Delta Lake and medallion/lakehouse architecture
Experience implementing incremental loads and CDC-style processing
Experience implementing data quality checks and troubleshooting pipeline failures
Python, PySpark, advanced SQL, Delta Lake, and distributed data processing
Microsoft Fabric: Data Factory pipelines and Copy Activity, Lakehouse, OneLake, Spark notebooks, Environments, Mirroring, and Shortcuts
Batch and incremental ingestion, CDC patterns, watermarking, reprocessing strategies, and schema-on-read for varied formats
Validation rules, completeness/accuracy checks, alerting, exception workflows, and reconciliation
Bronze/Silver/Gold medallion layering, cleansing and conformance, and standardization of names, addresses, dates, and codes
Pipeline monitoring, lineage and metadata capture, access controls, and technical documentation

Benefits

Comp & perks

Hybrid work arrangement
Full-time employment