Demonstrates expertise in designing and developing scalable ETL pipelines using Databricks, PySpark, and SQL, while ensuring data quality and governance. Proficient in implementing CI/CD practices and collaborating with stakeholders to deliver high-quality data solutions for AI and analytics.

Highest-signal resume keywords

Databricks Data EngineeringETL DevelopmentPython ProgrammingData Quality FrameworksMedallion Architecture

Tech Stack

Tools & technologies

AzureCloudETLPySparkPythonSparkSQL

About the role

Key responsibilities & impact

Design, develop, and maintain scalable batch and streaming ETL pipelines using Databricks, PySpark, Python, and SQL
Build and manage data solutions using Delta Lake and medallion architecture principles
Integrate data from enterprise applications, databases, APIs, files, and cloud platforms
Develop data models and transformation frameworks supporting business intelligence, advanced analytics, and AI/ML use cases
Monitor and optimize Databricks workloads, cluster configurations, and data processing performance
Implement data quality checks, monitoring, lineage, security controls, and governance practices
Contribute to CI/CD implementation, automated deployments, testing frameworks, and release management
Partner with business stakeholders, data architects, analytics teams, and AI engineers to deliver data products
Enable AI, machine learning, and Generative AI initiatives through reliable, high-quality, governed data foundations
Stay current with Databricks, cloud technologies, and modern data engineering practices to enhance platform capabilities

Requirements

What you’ll need

4–6 years of experience in data engineering, ETL development, and cloud-based data platforms
Proficiency in Python, PySpark, and SQL
Strong expertise in Databricks Data Engineering capabilities, including Databricks Lakehouse Architecture, ETL/ELT design and development, Delta Lake, data modeling, batch and streaming data pipelines, and Spark optimization and performance tuning
Experience designing and implementing Medallion Architecture, data quality frameworks, data governance controls, metadata, and lineage solutions
Experience with orchestration and workflow management tools such as Databricks Workflows, Azure Data Factory, or equivalent
Experience implementing CI/CD pipelines, Git-based source control, automated testing, and deployment automation
Strong understanding of data warehousing, dimensional modeling, and modern Lakehouse architectures
Exposure to Databricks Mosaic AI, Vector Search, MLflow, feature engineering, or GenAI data preparation workflows (Preferred)
Familiarity with data preparation for RAG, LLM, and AI/ML use cases (Preferred)
Experience with infrastructure-as-code and containerization technologies (Preferred)
Relevant data engineering certifications, such as Databricks Certified Data Engineer Associate/Professional, or similar (Preferred not mandatory)
Strong analytical and problem-solving skills, with the ability to design scalable data solutions and resolve complex data engineering challenges
Excellent communication skills with experience working directly with clients and business stakeholders
Experience engaging with business stakeholders to understand data and analytics requirements and translate them into scalable data engineering solutions
Bachelor's Degree

Benefits

Comp & perks

Hybrid work arrangement
Full-time employment

ATS Keywords

✓ Tailor your resume

Applicant Tracking System Keywords

Tip: use these terms in your resume and cover letter to boost ATS matches.

Hard Skills

ETL PipelinesDatabricksPySparkSQLDelta LakeData ModelingSpark OptimizationCI/CD ImplementationData GovernanceInfrastructure-as-Code

Soft Skills

Analytical SkillsProblem-SolvingCommunication Skills

Tools & Technologies

Databricks WorkflowsAzure Data FactoryGitMLflowContainerization Technologies

Certifications & Qualifications

Databricks Certified Data Engineer AssociateDatabricks Certified Data Engineer Professional

Industry Keywords

Data EngineeringCloud-Based Data PlatformsData WarehousingDimensional ModelingLakehouse ArchitectureAI/ML Use Cases