FREE ACCESS
5,000–10,000 jobs/day
See all jobs on Scoutfield
Search thousands of fresh jobs every day.
Discover
- Fresh listings
- Fast filters
- No subscription required
Create a free account and start exploring right away.
Core Competencies
Role fitCore Competencies
Use this summary to align your resume positioning with the role.
Demonstrates expertise in developing and implementing Data Engineering solutions using the Databricks ecosystem, with a strong focus on scalable data architectures, data governance, and performance optimization. Proficient in orchestrating data pipelines and integrating systems while mentoring team members in best practices.
Highest-signal resume keywords
Databricks Ecosystem ExperienceAdvanced PySpark ProficiencyData Architecture DefinitionAzure Data Factory (ADF) ExpertiseDatabricks Certified Data Engineer Associate
ATS Keywords
Tailor your resumeApplicant Tracking System Keywords
Tip: use these terms in your resume and cover letter to boost ATS matches.
Hard Skills
DatabricksPySparkSpark SQLDelta LakeData IngestionStructured StreamingREST APIs DevelopmentMicroservicesPerformance TuningData Governance
Soft Skills
MentoringTechnical ReferenceProblem SolvingAutonomy
Tools & Technologies
Azure Data FactoryDatabricks WorkflowsCI/CDGitLiquid Clustering
Certifications & Qualifications
Databricks Certified Data Engineer AssociateDatabricks Certified Data Engineer ProfessionalDatabricks Certified Associate Developer for Apache Spark
Industry Keywords
Lakehouse ModelMedallion ArchitectureData LineageData SecurityCost Optimization
Tech Stack
Tools & technologiesApacheAWSAzureGoogle Cloud PlatformPySparkPythonSparkSQLUnity
About the role
Key responsibilities & impact- Develop, implement, and evolve Data Engineering solutions using the Databricks ecosystem
- Define and implement scalable data architectures based on the Lakehouse model and Medallion Architecture (Bronze, Silver, and Gold)
- Develop, optimize, and maintain data pipelines using Databricks, PySpark, Spark SQL, and Delta Lake
- Implement data ingestion, processing, and integration solutions, including CDC and Structured Streaming
- Orchestrate pipelines and integrate systems using Azure Data Factory (ADF)
- Develop REST APIs, microservices, and automations using Python
- Ensure data governance, security, lineage, and access control
- Optimize performance, scalability, and processing costs
- Support observability, monitoring, and production operations for data pipelines
- Serve as a technical reference for the team, mentoring professionals and establishing Data Engineering best practices
Requirements
What you’ll need- Solid, advanced experience delivering production projects with Databricks
- Advanced proficiency in PySpark, Spark SQL, Delta Lake, and Apache Spark
- Experience with DLT / Lakeflow Spark Declarative Pipelines, CDC, and Structured Streaming
- Experience with Liquid Clustering, Auto Loader, and Delta table optimization
- Advanced knowledge of Unity Catalog, governance, security, lineage, and access control
- Experience defining data architectures, including the Lakehouse model and Medallion Architecture
- Experience with Azure Data Factory (ADF) for orchestration, ingestion, and data integration
- Proficiency in Python for developing REST APIs, microservices, and automations
- Experience with Spark performance tuning, Photon, partitioning, shuffle, and execution plan analysis
- Knowledge of Databricks Workflows, CI/CD, Git, and Declarative Automation Bundles
- Experience optimizing workloads, sizing compute resources, and reducing costs
- Experience with observability, monitoring, and production support for data pipelines
- Autonomy in resolving complex technical issues and making architectural decisions
- Experience serving as a technical reference, supporting and mentoring other professionals
- Required certification: Databricks Certified Data Engineer Associate
- Databricks Certified Data Engineer Professional certification strongly preferred
- Databricks Certified Associate Developer for Apache Spark certification is a plus
- Preferred: AWS, Azure, or GCP certifications related to Data Engineering and Solution Architecture
Benefits
Comp & perks- Meal and/or food allowance
- Health and dental insurance
- Health insurance for your pet
- Home office allowance (for remote positions)
- Welcome kit during onboarding
- Life insurance
- Zenklub, with psychological and nutritional support
- TotalPass
- Certification and Continuous Training Program with partners (Adobe, Anthropic, AWS, Google, Microsoft, Salesforce)
- Partnerships and discounts to support your professional development
- Access to UOL content
- Maternity kit
- Extended maternity and paternity leave
- Employee referral program