FREE ACCESS
5,000–10,000 jobs/day
See all jobs on Scoutfield
Search thousands of fresh jobs every day.
Discover
- Fresh listings
- Fast filters
- No subscription required
Create a free account and start exploring right away.
Core Competencies
Role fitCore Competencies
Use this summary to align your resume positioning with the role.
Demonstrates expertise in building and maintaining ETL pipelines using Python, PySpark, and SQL within Azure Databricks, while ensuring compliance with security controls and performance standards. Proven ability to collaborate effectively across teams and deliver high-quality data products in dynamic environments.
Highest-signal resume keywords
Python ProgrammingAzure DatabricksETL Pipeline DevelopmentSQL Data EngineeringAzure Data Factory
ATS Keywords
Tailor your resumeApplicant Tracking System Keywords
Tip: use these terms in your resume and cover letter to boost ATS matches.
Hard Skills
PySparkData EngineeringEvent-Driven Data IngestionUnit TestingIntegration TestingSecure ETL DesignData ModellingPerformance OptimizationData TransformationSOLID Principles
Soft Skills
Stakeholder CommunicationCross-Team CollaborationAnalytical MindsetOwnershipAdaptability
Tools & Technologies
Azure Data FactoryMicrosoft PurviewPower BIDatabricks Data Intelligence PlatformEvent GridPub-Sub
Certifications & Qualifications
BPSS Eligibility
Industry Keywords
UK Public SectorPlatform GovernanceAgile MethodologiesData GovernanceData Security
Tech Stack
Tools & technologiesAzureCloudETLPySparkPythonSQL
About the role
Key responsibilities & impact- Build and maintain ETL pipelines and code in cloud data platforms
- Support ingestion activity and onboarding of new data sources
- Work with the Databricks Data Intelligence Platform and develop data engineering workloads
- Construct raw, refined, and curated data layers; catalogue assets appropriately
- Validate solutions against functional and non-functional requirements
- Deliver datasets, transformations, and performance-optimised data products
- Improve processes, engineering patterns, and reusable tooling
- Monitor and measure pipeline performance; support incident resolution
- Ensure documentation meets acceptance standards and is approved centrally
- Actively engage in Agile ceremonies and governance forums
- Work closely with architects, analysts, engineers, and wider delivery teams
- Implement pipeline builds, data models, and integration patterns aligned to the organisation’s Target Operating Model for Data
- Develop scalable data flows and optimise ingestion and transformation activities
- Ensure solutions meet technical standards, performance expectations, and security controls
Requirements
What you’ll need- Strong experience with Python, PySpark, and SQL for data engineering
- Hands-on experience with Azure Databricks
- Strong knowledge of Azure Data Factory and Fabric
- Experience with Microsoft Purview
- Familiarity with event-driven data ingestion (Event Grid / Pub-Sub)
- Understanding of SOLID principles, async programming, and Mediator/Factory patterns
- Experience delivering unit and integration testing in Databricks
- Knowledge of secure ETL design with Entra IMID/SCIM integration
- Understanding of Azure best practices, APIM, and platform governance
- Ability to build and serve Power BI models via Databricks data sources
- Prior experience working within UK Public Sector environments desirable
- Candidates must be eligible to work with UK Government departments / BPSS
- Strong stakeholder communication and cross-team collaboration
- Analytical and solution-focused mindset
- Ability to work independently, take ownership, and drive progress
- Commitment to clean, scalable, well-documented engineering
- Adaptable, proactive, and comfortable working in dynamic delivery environments
Benefits
Comp & perks- Technical autonomy, ownership, and challenge
- Remote work arrangement for UK-based candidates
- Contract/freelance engagement
- ASAP start