FREE ACCESS
5,000–10,000 jobs/day
See all jobs on Scoutfield
Search thousands of fresh jobs every day.
Discover
- Fresh listings
- Fast filters
- No subscription required
Create a free account and start exploring right away.
Core Competencies
Role fitCore Competencies
Use this summary to align your resume positioning with the role.
Demonstrates expertise in designing and building data pipelines using Azure Databricks, PySpark, and Spark SQL, with a strong focus on dimensional modeling and data warehousing best practices. Proficient in implementing incremental data processing and ensuring data accuracy through reconciliation with source systems.
Highest-signal resume keywords
Azure DatabricksPySparkSpark SQLDimensional ModelingData Warehousing
ATS Keywords
Tailor your resumeApplicant Tracking System Keywords
Tip: use these terms in your resume and cover letter to boost ATS matches.
Hard Skills
Data EngineeringDelta LakeIncremental Data ProcessingChange Data CapturePerformance OptimizationRelational DatabasesSQLStored ProceduresScalable Production PipelinesData Reconciliation
Tools & Technologies
LakeflowSpark Declarative PipelinesDbtUnity CatalogJD Edwards
Industry Keywords
Financial Data ModelsGeneral Ledger Data Models
Tech Stack
Tools & technologiesAzureDistributed SystemsPySparkSparkSQLUnity
About the role
Key responsibilities & impact- Design and build conformed dimensions and transactional fact tables for enterprise reporting
- Develop and maintain data pipelines on Azure Databricks using PySpark, Spark SQL, and Delta Lake
- Implement dimensional modeling best practices, including star schemas, conformed dimensions, surrogate and deterministic hash keys such as SHA-256, and transactional fact design
- Translate existing SQL Server and stored procedure logic into efficient Spark-based transformations
- Implement incremental data processing and change data capture (CDC) pipelines
- Ensure data reconciliation with source systems to meet accuracy and audit standards
- Collaborate closely with a senior Databricks lead to deliver core data assets within defined timelines
Requirements
What you’ll need- 8+ years of experience in data engineering, with strong focus on Azure Databricks
- Proficiency in PySpark
- Proficiency in Spark SQL
- Proficiency in Delta Lake
- Deep expertise in dimensional modelling and data warehousing concepts
- Experience building scalable production pipelines
- Experience with incremental data loads and CDC patterns
- Experience with performance optimization in distributed systems
- Strong understanding of relational databases
- Ability to interpret complex SQL/stored procedures
- Any Degree
- Experience with Lakeflow / Spark Declarative Pipelines or dbt (good to have)
- Experience with Unity Catalog (good to have)
- Exposure to financial/General Ledger data models (good to have)
- Familiarity with JD Edwards systems, including F0902 and F0911 (good to have)