Senior Databricks Engineer

EXL

. Design, develop, and maintain data pipelines using Databricks, Apache Spark, and cloud-native services .

Posted 10/15/2026full-timeBengaluru • IndiaSeniorWebsite

Core Competencies

Role fit

Core Competencies

Use this summary to align your resume positioning with the role.

Demonstrates expertise in designing and optimizing data pipelines using Databricks and Apache Spark, with a strong focus on data governance, quality, and integration within cloud-native architectures. Proficient in implementing CI/CD practices and collaborating with cross-functional teams to deliver high-quality data products.

Highest-signal resume keywords

Azure Data EngineeringDatabricks Lakehouse PlatformApache Spark/PySparkETL/ELT DevelopmentData Governance Frameworks

ATS Keywords

Tailor your resume

Applicant Tracking System Keywords

Tip: use these terms in your resume and cover letter to boost ATS matches.

Hard Skills

Data Pipeline DevelopmentData ModelingSQLPythonDelta LakeStreaming ProcessingData Quality ValidationCloud-Native ArchitecturePerformance OptimizationData Integration

Soft Skills

Analytical SkillsProblem-SolvingStakeholder ManagementCommunicationTeam Leadership

Tools & Technologies

AzureAWSGCPTerraformGitHub ActionsAzure DevOpsCI/CD PipelinesUnity CatalogAgile/ScrumBI Platforms

Certifications & Qualifications

Bachelor's DegreeMaster's Degree

Industry Keywords

Data LakeLakehouse ArchitectureData GovernanceData LineageAccess ControlComplianceReal-Time ProcessingMachine LearningGenerative AIData Products

Tech Stack

Tools & technologies

ApacheAWSAzureCloudETLGoogle Cloud PlatformPySparkPythonSparkSQLTerraformUnity

About the role

Key responsibilities & impact

Design, develop, and maintain data pipelines using Databricks, Apache Spark, and cloud-native services
Build and optimize ETL/ELT workflows for large-scale structured and unstructured data
Develop data models and implement data quality, validation, and governance frameworks
Integrate data from multiple sources into a unified Lakehouse architecture
Optimize Spark jobs and Databricks workloads for performance, scalability, and cost efficiency
Implement security controls, access management, and data governance using Unity Catalog
Collaborate with business, analytics, and AI/ML teams to deliver trusted data products
Monitor, troubleshoot, and resolve data pipeline issues
Support CI/CD, DevOps, and infrastructure automation practices
Maintain technical documentation and best practices

Requirements

What you’ll need

9–15 years of IT experience
Minimum 5+ years of hands-on experience with Azure Data Engineering technologies
Minimum 4+ years of hands-on experience with Azure Databricks and Spark ecosystem
Bachelor's or Master's degree in Computer Science, Information Technology, Engineering, or related field
Strong understanding of data lake, lakehouse, and cloud-native architecture patterns
Experience handling large-scale structured and unstructured datasets
Strong analytical, problem-solving, and troubleshooting skills
Proficiency in Databricks Lakehouse Platform, Apache Spark/PySpark, Delta Lake, SQL, and Python
Experience with ETL/ELT development, data modeling, data warehousing, data quality and validation, and streaming/real-time processing
Knowledge of Unity Catalog, data lineage, row-level security, access control, compliance, and data governance frameworks
Experience with Azure, AWS, or GCP, Terraform, GitHub Actions or Azure DevOps, and CI/CD pipelines
Experience with semantic layers, data products, BI platforms, machine learning support, and generative AI/RAG architectures
Experience working in Agile/Scrum environments
Strong stakeholder management, communication, documentation, presentation, mentoring, and team leadership abilities