FREE ACCESS
5,000–10,000 jobs/day
See all jobs on Scoutfield
Search thousands of fresh jobs every day.
Discover
- Fresh listings
- Fast filters
- No subscription required
Create a free account and start exploring right away.
Core Competencies
Role fitCore Competencies
Use this summary to align your resume positioning with the role.
Demonstrates strong expertise in designing and optimizing ETL/ELT pipelines using Databricks, PySpark, and SQL, while ensuring data quality and performance. Capable of leading technical decisions and collaborating with cross-functional teams to deliver scalable data solutions.
Highest-signal resume keywords
PythonPySparkAdvanced SQLDatabricks Lakehouse PlatformETL/ELT Development
ATS Keywords
Tailor your resumeApplicant Tracking System Keywords
Tip: use these terms in your resume and cover letter to boost ATS matches.
Hard Skills
Data ModelingDimensional Data ModelsData TransformationBatch ProcessingIncremental LoadingData WarehousingSpark Performance TuningQuery OptimizationData Quality ValidationREST API Integration
Soft Skills
CollaborationTechnical Leadership
Tools & Technologies
DatabricksUnity CatalogDelta LakeDatabricks WorkflowsGitCI/CD
Certifications & Qualifications
Databricks Certification
Industry Keywords
Medallion ArchitectureStar SchemaSnowflake SchemaFact TablesDimension TablesSCD ConceptsStructured DataSemi-Structured DataCSVJSON
Tech Stack
Tools & technologiesAirflowApacheETLKafkaPySparkPythonSparkSQLUnity
About the role
Key responsibilities & impact- Design, develop, and maintain scalable ETL/ELT pipelines using Databricks, PySpark, and SQL
- Integrate data from multiple sources, including databases, Amazon S3, files, and REST APIs
- Build data pipelines with Databricks Unity Catalog
- Implement business logic, data transformations, and dimensional data models
- Create, schedule, monitor, and optimize Databricks Jobs and Workflows
- Design and manage Delta Lake tables using Medallion Architecture (Bronze, Silver, Gold)
- Ensure data quality through validations, error handling, logging, and monitoring
- Optimize Spark workloads for performance, scalability, and reliability
- Collaborate with cross-functional teams to deliver production-ready data solutions
- Guide engineers and drive technical decisions in a lead capacity
Requirements
What you’ll need- Strong expertise in Python, PySpark, and Advanced SQL
- Hands-on experience with the Databricks Lakehouse Platform
- Good understanding of Unity Catalog, Delta Lake, Databricks Workflows/Jobs, Clusters, Notebooks, Repos, and Medallion Architecture
- Experience integrating with REST APIs for data ingestion and data export
- Strong knowledge of ETL/ELT development, batch processing, incremental loading, and data transformation
- Experience with data modeling, including Star Schema, Snowflake Schema, Fact & Dimension tables, and SCD concepts
- Understanding of data warehousing concepts and best practices
- Experience working with structured and semi-structured data, including CSV, JSON, Parquet, and Delta
- Knowledge of partitioning, file optimization, Spark performance tuning, and query optimization
- Experience with Git and CI/CD best practices
- 9+ years of experience in Data Engineering, including 3+ years of hands-on experience with Databricks
- Prior experience in a Lead Data Engineer / Technical Lead role, guiding engineers and driving technical decisions
- Experience designing, developing, and optimizing ETL/ELT data pipelines
- Experience with Apache Spark and SQL
- Experience with Auto Loader, Spark Declarative Pipelines, Kafka, Airflow, or dbt is a plus
- Databricks certification is an added advantage
Benefits
Comp & perks- Competitive salary
- Strong insurance package
- Extensive learning and development resources