Consultant – PySpark, Databricks Developer, Intermediate – Senior

Electric Mind

. Build and maintain Databricks data pipelines for a financial services/wealth management data platform .

Posted 10/9/2026full-timeToronto • CanadaSenior💰 CA$100,000 - CA$140,000 per yearWebsite

Core Competencies

Role fit

Core Competencies

Use this summary to align your resume positioning with the role.

Proficient in building and maintaining Databricks data pipelines, with strong skills in PySpark and SQL for data transformation and validation. Capable of debugging and optimizing data workflows while ensuring high data quality and documentation standards.

Highest-signal resume keywords

Databricks Data Pipeline DevelopmentPySpark Transform WritingSQL Querying and ValidationGit Workflow ManagementData Quality Debugging

ATS Keywords

Tailor your resume

Applicant Tracking System Keywords

Tip: use these terms in your resume and cover letter to boost ATS matches.

Hard Skills

Python ProgrammingSQLPySparkDatabricksDelta LakeCI/CDData Pipeline DebuggingUnit TestingIntegration TestingData Quality Assurance

Soft Skills

Quick LearnerEffective CommunicationProblem Solving

Tools & Technologies

GitLabUnity CatalogPowerShellAzureKafka

Industry Keywords

Financial ServicesWealth ManagementCDCSCDMedallion Architecture

Tech Stack

Tools & technologies

ApacheAzureKafkaPySparkPythonSparkSQLUnity

About the role

Key responsibilities & impact

Build and maintain Databricks data pipelines for a financial services/wealth management data platform
Write and maintain hand-written PySpark transforms for silver- and gold-layer customer, account, and transaction tables
Work with Lakeflow Declarative Pipelines and Auto CDC/SCD Type 2 patterns
Query and validate data in Unity Catalog across dev/qa/uat using SQL warehouses
Debug failed pipeline runs by reading event logs, tracing bad records, and fixing schema drift or data quality issues
Maintain reference/lookup tables and PowerShell/Python utility scripts
Write and update unit and integration tests
Participate in GitLab code reviews using feature branches and merge requests
Keep technical documentation current
Work alongside a senior engineer and help maintain healthy development, QA, and UAT environments
In the first 90 days, independently handle small transform bugs or enhancements, operational scripts, pipeline diagnosis, and begin taking ownership of Databricks environments and new transforms

Requirements

What you’ll need

Comfortable writing clean, readable Python code; understands functions, modules, and basic OOP
Databricks knowledge
Some exposure to Apache Spark/PySpark, or strong SQL skills plus willingness to learn Spark quickly
Working knowledge of SQL, including joins, aggregations, and window functions
Basic Git workflow: branches, commits, and pull/merge requests
Comfortable reading others' code and stack traces and debugging methodically
Direct Databricks experience, Delta Lake, medallion architecture, CDC/SCD concepts, Azure, Kafka, CI/CD, and financial services experience are nice-to-have
Must be able to learn quickly, ask good questions, and write clean code