Senior Data Engineer – Databricks, PySpark, SQL

HumanIT Digital Consulting

. Design, develop and maintain scalable data ingestion and transformation pipelines on a Databricks-based Lakehouse platform .

Posted 10/23/2026full-timeRemote • Portugal, SpainSenior💰 €2,300 - €3,050 per monthWebsite

Core Competencies

Role fit

Core Competencies

Use this summary to align your resume positioning with the role.

Demonstrates expertise in designing and maintaining scalable data ingestion and transformation pipelines using Databricks, PySpark, and SQL. Proficient in implementing data governance practices and collaborating with cross-functional teams to deliver business-ready data solutions.

Highest-signal resume keywords

Databricks DevelopmentPySpark Data ProcessingSQL Development and OptimisationETL/ELT Pipeline DesignAzure Data Platform Services

ATS Keywords

Tailor your resume

Applicant Tracking System Keywords

Tip: use these terms in your resume and cover letter to boost ATS matches.

Hard Skills

Data EngineeringData IngestionData TransformationPipeline OptimisationData Quality PrinciplesData Security PrinciplesReusable Engineering FrameworksMedallion ArchitectureDelta Live TablesData Lineage

Soft Skills

CollaborationEffective Communication

Tools & Technologies

DatabricksGitCI/CD PracticesAzure Data PlatformMicrosoft Fabric

Industry Keywords

Lakehouse ArchitectureData GovernanceBusiness IntelligenceAnalyticsEnterprise Data Platform

Tech Stack

Tools & technologies

AzureCloudETLPySparkSQLUnity

About the role

Key responsibilities & impact

Design, develop and maintain scalable data ingestion and transformation pipelines on a Databricks-based Lakehouse platform
Implement Bronze, Silver and Gold data layers
Integrate data from enterprise systems, APIs and external sources
Develop processing workflows using PySpark and SQL
Implement reusable engineering frameworks
Optimise pipelines for scalability and cost efficiency
Translate established architecture standards into practical engineering solutions
Support data governance, lineage, cataloguing and access management initiatives
Enable BI and Analytics teams to consume trusted, business-ready data
Participate in Agile ceremonies
Contribute to delivery through Git-based version control and CI/CD practices
Collaborate with Data Architects, Analytics teams and business stakeholders

Requirements

What you’ll need

At least 4 years of professional Data Engineering experience
At least 2 years of hands-on Databricks development experience
Strong practical skills in Databricks and PySpark for building and maintaining data processing pipelines
Strong SQL development and optimisation skills
Experience with Azure Data Platform services
Proven experience designing and implementing ETL/ELT pipelines, including data ingestion and transformation
Understanding of Lakehouse architecture and Medallion Architecture, including Bronze, Silver and Gold data layers
Experience integrating data from multiple enterprise systems, APIs and external sources
Knowledge of data quality, pipeline reliability, performance optimisation and data security principles
Experience using Git and CI/CD practices in data engineering workflows
Ability to collaborate effectively with Data Architects, BI and Analytics teams, and business stakeholders
Professional working proficiency in English
Availability to work remotely from Portugal or Spain
Preferred: Experience with Delta Live Tables (DLT)
Preferred: Familiarity with Unity Catalog and Databricks data governance capabilities
Preferred: Experience with Microsoft Fabric
Preferred: Previous involvement in Data Hub or Enterprise Data Platform programmes
Preferred: Experience developing reusable data engineering frameworks and components
Preferred: Knowledge of data lineage, cataloguing and enterprise data governance practices
Preferred: Experience optimising cloud data workloads for performance and cost efficiency

Benefits

Comp & perks

15th month salary
Health insurance covering your family
Birthday off
Mobility program for digital nomads
Real work-life balance