FREE ACCESS
5,000–10,000 jobs/day
See all jobs on Scoutfield
Search thousands of fresh jobs every day.
Discover
- Fresh listings
- Fast filters
- No subscription required
Create a free account and start exploring right away.
Senior Data Engineer
Unifeye Vision Partners. Design and build scalable batch and streaming data pipelines using Spark and Databricks.
Core Competencies
Role fitCore Competencies
Use this summary to align your resume positioning with the role.
Demonstrates expertise in designing and building scalable data pipelines using Apache Spark and Databricks, with a strong focus on Lakehouse architecture and data governance. Proficient in optimizing data workloads for performance and cost efficiency while collaborating effectively with stakeholders.
Highest-signal resume keywords
Apache Spark (Pyspark)DatabricksSQLLakehouse ArchitectureCI/CD Implementation
ATS Keywords
Tailor your resumeApplicant Tracking System Keywords
Tip: use these terms in your resume and cover letter to boost ATS matches.
Hard Skills
Data Pipeline DevelopmentData ModellingStreaming TechnologiesData Quality AssurancePerformance Optimization
Soft Skills
Strong CommunicationCollaboration with Stakeholders
Tools & Technologies
Delta LakeUnity CatalogTerraformKafkaMLOps
Certifications & Qualifications
Databricks Certification
Industry Keywords
Data GovernanceCloud Platforms (AWS, Azure, GCP)Infrastructure-as-CodeClient-Facing DeliveryAnalytics Integration
Tech Stack
Tools & technologiesApacheAWSAzureGoogle Cloud PlatformKafkaPySparkSparkSQLTerraformUnity
About the role
Key responsibilities & impact- Design and build scalable batch and streaming data pipelines using Spark and Databricks.
- Develop modern Lakehouse architectures using Delta Lake and Medallion (Bronze, Silver and Gold) design principles.
- Build robust ingestion frameworks from APIs, files and streaming data sources.
- Ensure data quality, testing, monitoring and observability across data pipelines.
- Implement governance, access control and lineage using Unity Catalog.
- Optimise workloads for performance, scalability and cost efficiency.
- Collaborate with architects and customers to translate business requirements into production-ready data solutions.
- Support integration with downstream analytics, BI, machine learning and AI workloads.
- Contribute to CI/CD pipelines, deployment automation and infrastructure-as-code practices.
- Share knowledge across the team and support the continued growth of Unifeye.
Requirements
What you’ll need- Strong commercial experience building production-grade data pipelines using Apache Spark (PySpark preferred).
- Experience delivering modern data platforms using Databricks or an equivalent Spark-based platform.
- Strong SQL skills and experience with data modelling.
- Good understanding of Lakehouse architecture and Delta Lake.
- Experience working with AWS, Azure or GCP.
- Experience implementing CI/CD, version control and deployment automation.
- Strong communication skills and experience working with technical and business stakeholders.
- Experience with Unity Catalog or similar data governance frameworks.
- Experience with Terraform or Databricks Asset Bundles.
- Exposure to MLflow, MLOps or AI and Generative AI workloads.
- Experience with streaming technologies such as Kafka or Structured Streaming.
- Databricks certification.
- Previous consulting or client-facing delivery experience.
Benefits
Comp & perks- The opportunity to work exclusively with Databricks technologies.
- High levels of ownership and autonomy from day one.
- The chance to make a real impact on clients and the growth of the business.
- Close collaboration with Databricks teams.
- Continuous learning, certification and career development.
- Competitive salary, bonus, generous benefits and flexible hybrid working.