FREE ACCESS
5,000–10,000 jobs/day
See all jobs on Scoutfield
Search thousands of fresh jobs every day.
Discover
- Fresh listings
- Fast filters
- No subscription required
Create a free account and start exploring right away.
Core Competencies
Role fitCore Competencies
Use this summary to align your resume positioning with the role.
Demonstrates expertise in designing and implementing scalable data pipelines and Lakehouse architectures, with a strong focus on data governance, security, and performance optimization. Proven ability to lead technical delivery and collaborate effectively with cross-functional teams to deliver end-to-end data solutions.
Highest-signal resume keywords
DatabricksPySparkETL/ELT ProcessesData GovernanceLakehouse Architecture
ATS Keywords
Tailor your resumeApplicant Tracking System Keywords
Tip: use these terms in your resume and cover letter to boost ATS matches.
Hard Skills
Apache SparkDelta LakeData ModellingData Lifecycle ManagementPerformance OptimizationData Quality ControlCloud Data PlatformsData Engineering Best PracticesTechnical Delivery LeadershipSC Security Clearance Eligibility
Soft Skills
Stakeholder EngagementCommunication SkillsMentoring
Tools & Technologies
Unity CatalogAzureAWS
Industry Keywords
Data PipelinesMedallion Design PrinciplesOperational SupportData ArchitectureCloud-Native Services
Tech Stack
Tools & technologiesApacheAWSAzureCloudETLPySparkSparkSQLUnity
About the role
Key responsibilities & impact- Design, build and optimise scalable data pipelines using Databricks, Apache Spark, PySpark and Delta Lake
- Develop and implement Lakehouse architectures using Medallion design principles
- Build ETL and ELT processes to ingest, transform and curate large, complex and diverse datasets
- Drive best practice in data quality, monitoring, alerting and operational support
- Optimise platform performance, scalability and cost across compute, storage and orchestration services
- Contribute to enterprise data architectures, standards and engineering patterns
- Integrate data platforms with cloud-native services, data lakes, event streams, warehouses and enterprise systems
- Implement data governance, security and access controls, including Unity Catalog or equivalent technologies
- Lead technical delivery across data engineering workstreams
- Collaborate with data scientists, analysts, architects and business stakeholders on end-to-end data solutions
- Support DevOps and CI/CD practices for data engineering
- Stay current with emerging technologies and industry best practice
- Mentor team members and influence the future direction of data engineering capability
Requirements
What you’ll need- Proven hands-on experience building and supporting production-grade data pipelines using Databricks, PySpark, Spark SQL and Delta Lake
- Experience designing and implementing Lakehouse architectures, including Medallion (Bronze, Silver, Gold) data models
- Expertise in scalable ETL/ELT pipelines handling large and complex datasets
- Understanding of modern cloud data platforms, particularly Azure and/or AWS
- Experience implementing data governance, security and quality controls, including Unity Catalog or equivalent technologies
- Strong knowledge of data modelling, data lifecycle management and data engineering best practices
- Experience optimising data workloads for performance, scalability, reliability and cost efficiency
- Ability to make sound technical and architectural decisions within complex delivery environments
- Strong stakeholder engagement and communication skills, with ability to explain technical concepts to technical and non-technical audiences
- Experience leading technical delivery and supporting successful execution of data engineering projects
- Must be eligible to obtain and maintain SC security clearance
Benefits
Comp & perks- 25 days annual leave plus your birthday off
- Ability to buy or sell up to five additional days
- Private healthcare with additional options for family members
- Ongoing learning opportunities and training
- Fitness reimbursement for gym memberships or fitness-related expenses
- Life assurance coverage
- Competitive pension contribution options
- Perkbox subscription with discounts on products and services
- Flexible work arrangements
- Internal reward schemes and recognition initiatives
- Community engagement and volunteering opportunities through company-sponsored programs