FREE ACCESS
5,000–10,000 jobs/day
See all jobs on Scoutfield
Search thousands of fresh jobs every day.
Discover
- Fresh listings
- Fast filters
- No subscription required
Create a free account and start exploring right away.
Core Competencies
Role fitCore Competencies
Use this summary to align your resume positioning with the role.
Demonstrates expertise in designing and optimizing data pipelines using Databricks and Apache Spark, with a strong focus on data governance, quality, and integration within cloud-native architectures. Proficient in implementing CI/CD practices and collaborating with cross-functional teams to deliver high-quality data products.
Highest-signal resume keywords
Azure Data Engineering TechnologiesDatabricks Lakehouse PlatformApache Spark / PySparkETL/ELT DevelopmentData Governance Frameworks
ATS Keywords
Tailor your resumeApplicant Tracking System Keywords
Tip: use these terms in your resume and cover letter to boost ATS matches.
Hard Skills
Data EngineeringData WarehousingData ModelingSQLPythonStreaming ProcessingDelta LakeETL/ELT WorkflowsData QualityData Validation
Soft Skills
Analytical SkillsProblem-SolvingTroubleshooting
Tools & Technologies
DatabricksApache SparkUnity CatalogTerraformGitHub ActionsAzure DevOpsCI/CD PipelinesPower BIMicrosoft FabricKafka/Event Hubs
Certifications & Qualifications
Azure Data Engineer Associate (DP-203)Databricks Certified Data Engineer
Industry Keywords
Cloud-Native ArchitectureLakehouse ArchitectureData GovernanceAgile/ScrumData Lineage
Tech Stack
Tools & technologiesApacheAWSAzureCloudETLGoogle Cloud PlatformKafkaPySparkPythonSparkSQLTerraformUnity
About the role
Key responsibilities & impact- Design, develop, and maintain data pipelines using Databricks, Apache Spark, and cloud-native services
- Build and optimize ETL/ELT workflows for large-scale structured and unstructured data
- Develop data models and implement data quality, validation, and governance frameworks
- Integrate data from multiple sources into a unified Lakehouse architecture
- Optimize Spark jobs and Databricks workloads for performance, scalability, and cost efficiency
- Implement security controls, access management, and data governance using Unity Catalog
- Collaborate with business, analytics, and AI/ML teams to deliver trusted data products
- Monitor, troubleshoot, and resolve data pipeline issues
- Support CI/CD, DevOps, and infrastructure automation practices
- Maintain technical documentation and best practices
Requirements
What you’ll need- Bachelor's or Master's degree in Computer Science, Information Technology, Engineering, or related field
- 9-15 years of experience in Data Engineering and Data Warehousing
- Minimum 5+ years of hands-on experience with Azure Data Engineering technologies
- Minimum 4+ years of hands-on experience with Azure Databricks and Spark ecosystem
- Strong understanding of data lake, lakehouse, and cloud-native architecture patterns
- Experience handling large-scale structured and unstructured datasets
- Strong analytical, problem-solving, and troubleshooting skills
- Databricks Lakehouse Platform
- Apache Spark / PySpark
- Delta Lake
- SQL
- Python
- ETL/ELT development, data modeling, data warehousing, data quality and validation
- Streaming and real-time processing
- Unity Catalog, data lineage, row-level security, access control and compliance, and data governance frameworks
- Azure, AWS, or GCP
- Terraform
- GitHub Actions or Azure DevOps
- CI/CD pipelines
- Semantic layers, data products, BI platforms, machine learning support, and generative AI/RAG architectures
- Experience working in Agile/Scrum environments
- Preferred certifications and experience include Azure Data Engineer Associate (DP-203), Databricks Certified Data Engineer, Snowflake, Power BI, Microsoft Fabric, Kafka/Event Hubs, and data governance initiatives
Benefits
Comp & perks- No benefits, perks, or compensation extras are specified in the posting