Senior Data Engineer – Databricks

INDT - Instituto de Desenvolvimento Tecnológico

. Implement and enhance the Databricks data platform .

Posted 9/22/2026full-timeRemote • BrazilSeniorWebsite

Core Competencies

Role fit

Core Competencies

Use this summary to align your resume positioning with the role.

Demonstrates expertise in implementing and enhancing the Databricks data platform, with a strong focus on data pipeline development using Spark/PySpark and Delta Lake. Proficient in optimizing data processing and ensuring data quality, governance, and observability in high-volume environments.

Highest-signal resume keywords

DatabricksApache SparkPySparkDelta LakeData Engineering Best Practices

ATS Keywords

Tailor your resume

Applicant Tracking System Keywords

Tip: use these terms in your resume and cover letter to boost ATS matches.

Hard Skills

Data Pipeline DevelopmentData ArchitecturePerformance OptimizationData ProcessingData Quality

Soft Skills

Problem AnalysisIndependent WorkCollaboration

Tools & Technologies

Unity CatalogAWSTerraformInfrastructure as CodeKafka

Certifications & Qualifications

Databricks Certification

Industry Keywords

Data GovernanceObservability ToolsHigh-Availability RequirementsLarge-Scale ProjectsCorporate Environments

Tech Stack

Tools & technologies

ApacheAWSKafkaPySparkSparkTerraformUnity

About the role

Key responsibilities & impact

Implement and enhance the Databricks data platform
Develop and enhance data pipelines using Spark/PySpark
Build ingestion, transformation, and processing solutions for large volumes of data
Work with Delta Lake and resources across the Databricks ecosystem
Contribute to the architecture and modeling of scalable data solutions
Implement data engineering best practices related to quality, governance, and observability
Identify and resolve performance issues in data pipelines and processes
Optimize processing, storage, and queries
Work with Unity Catalog and related data governance and security capabilities
Participate in defining and implementing pipeline orchestration and integration strategies
Collaborate with technology and business teams to define the best technical solutions
Ensure the quality, reliability, scalability, and maintainability of developed solutions
Contribute to the advancement of the project's data engineering standards and best practices

Requirements

What you’ll need

Solid, hands-on experience with Databricks
Advanced experience with Apache Spark and PySpark
Experience with Delta Lake
Knowledge of and experience with Unity Catalog
Experience implementing and enhancing data pipelines
Knowledge of data architecture and data engineering best practices
Experience optimizing and troubleshooting performance in Spark/Databricks
Experience processing large volumes of data
Ability to work independently when analyzing problems and defining technical solutions
Experience in corporate/enterprise environments
Preferred: Databricks certification
Preferred: Experience with AWS
Preferred: Experience with Terraform and Infrastructure as Code (IaC) practices
Preferred: Knowledge of or experience with Kafka and other streaming technologies
Preferred: Experience with data quality
Preferred: Knowledge of observability tools and practices
Preferred: Previous experience on large-scale projects
Preferred: Experience working in high-volume environments with high-availability requirements

Benefits

Comp & perks

Collaborative environment
Learning and career growth opportunities
Excellent workplace
Inspiring colleagues and leaders
Freedom to propose and develop innovative projects
Autonomy and ownership
Opportunities to develop your talents
An environment where you feel valued