FREE ACCESS
5,000–10,000 jobs/day
See all jobs on Scoutfield
Search thousands of fresh jobs every day.
Discover
- Fresh listings
- Fast filters
- No subscription required
Create a free account and start exploring right away.
Core Competencies
Role fitCore Competencies
Use this summary to align your resume positioning with the role.
Demonstrates expertise in Data Engineering with a focus on Databricks and Azure, including the design and optimization of scalable data pipelines. Proficient in DataOps practices and technical leadership, ensuring adherence to coding standards and governance policies.
Highest-signal resume keywords
Data EngineeringDatabricksPySparkAzure Data ServicesDataOps
ATS Keywords
Tailor your resumeApplicant Tracking System Keywords
Tip: use these terms in your resume and cover letter to boost ATS matches.
Hard Skills
DatabricksPySparkSpark SQLSQLPythonDelta LakeDimensional ModelingCI/CDAutomated TestingPerformance Optimization
Soft Skills
Technical LeadershipMentoringCollaborationProblem-SolvingCommunication
Tools & Technologies
Azure DatabricksAzure Data Lake StorageGitEvent HubsKafkaUnity CatalogDatabricks WorkflowsDelta Live TablesGreat ExpectationsSoda
Certifications & Qualifications
Databricks CertificationAzure Certification
Industry Keywords
DataOpsMedallion ArchitectureBronze/Silver/Gold LayersGovernanceAccess Control
Tech Stack
Tools & technologiesAirflowAzureKafkaPySparkPythonSparkSQLUnity
About the role
Key responsibilities & impact- Drive technical leadership of data engineering initiatives on Databricks in Azure
- Partner with the Data Architect to define standards and references, remove roadblocks, and enable scalable, reliable pipelines within the Medallion architecture
- Provide technical leadership to Data Engineering squads through design, reviews, and decision-making
- Co-define the technical vision, coding standards, modeling conventions, and Lakehouse best practices with the Data Architect
- Design and oversee Databricks pipelines across Bronze, Silver, and Gold layers
- Implement and reinforce DataOps practices including Git, CI/CD, unit/integration/data-quality testing, and documentation
- Optimize jobs and pipelines through partitioning, Spark tuning, cost management, observability, and alerting
- Ensure security, access control, and data policies in Azure/Databricks align with governance
- Collaborate with business and analytics stakeholders to translate requirements into scalable solutions and realistic delivery plans
- Mentor engineers, promote standards, and create templates and playbooks to accelerate the team
Requirements
What you’ll need- Strong experience in Data Engineering and projects, with experience as a Tech Lead/Senior leading teams technically and operationally
- Strong hands-on experience in Databricks, including PySpark and Spark SQL
- Experience with batch pipelines and working knowledge of streaming when needed
- Mastery of Delta Lake and practical application of the Medallion architecture (Bronze/Silver/Gold)
- Solid experience with Azure data services, including Azure Databricks, Azure Data Lake Storage (ADLS), and integration with security/identity services such as Azure Active Directory
- Strong SQL and Python skills
- Fundamentals in dimensional/consumption modeling and good practices for contracts/schemas
- Experience with DataOps, Git, CI/CD for data pipelines, automated testing, and clear documentation
- Proven ability to review design/code, resolve performance/scale issues, and guide teams
- Advanced English to work with stakeholders and produce technical documentation
- Nice to have: Unity Catalog, Databricks Workflows/Repos, Delta Live Tables
- Nice to have: Great Expectations/Soda and/or dbt on the Lakehouse
- Nice to have: Airflow/Databricks Workflows and observability through metrics, logs, and alerts
- Nice to have: CDC/streaming ingestion with Event Hubs/Kafka
- Nice to have: Consulting experience, international client exposure, and Databricks/Azure certifications
Benefits
Comp & perks- Health and dental insurance
- Meal and food allowance
- Childcare assistance
- Extended paternity leave
- Partnership with gyms and health and wellness professionals via Wellhub (Gympass) TotalPass
- Profit Sharing and Results Participation (PLR)
- Life insurance
- Continuous learning platform (CI&T University)
- Discount club
- Free online platform dedicated to physical, mental, and overall well-being
- Pregnancy and responsible parenting course
- Partnerships with online learning platforms
- Language learning platform
- Health and Well-being team, inclusion specialists, and affinity groups
- Support and accommodations for persons with disabilities during the selection process
