PySpark/Databricks Developer, Junior to Intermediate

Electric Mind

. Write and maintain hand-written PySpark transforms for silver and gold layer tables, including customer, account, and transaction data models .

Posted 9/18/2026full-timeToronto • CanadaJuniorWebsite

Core Competencies

Role fit

Core Competencies

Use this summary to align your resume positioning with the role.

Demonstrates expertise in writing and maintaining PySpark transforms and building Databricks data pipelines for financial services. Proficient in SQL querying, debugging, and maintaining data quality across various environments.

Highest-signal resume keywords

PySpark DevelopmentSQL QueryingDatabricks Data PipelinesGit WorkflowData Quality Management

ATS Keywords

Tailor your resume

Applicant Tracking System Keywords

Tip: use these terms in your resume and cover letter to boost ATS matches.

Hard Skills

Python ProgrammingSQLApache SparkUnit TestingIntegration Testing

Soft Skills

DebuggingCode ReviewDocumentation

Tools & Technologies

DatabricksLakeflowUnity CatalogPowerShellKafka

Industry Keywords

Financial ServicesData QualityChange Data CaptureSCD Type 2Medallion Architecture

Tech Stack

Tools & technologies

ApacheAzureKafkaPySparkPythonSparkSQLUnity

About the role

Key responsibilities & impact

Write and maintain hand-written PySpark transforms for silver and gold layer tables, including customer, account, and transaction data models
Work with Lakeflow Declarative Pipelines and Auto CDC / SCD Type 2 patterns for change-data-capture merges
Query and validate data in Unity Catalog across dev/qa/uat environments using SQL warehouses
Debug failed pipeline runs by reading event logs, tracing bad records, and fixing schema drift or data quality issues
Maintain reference/lookup tables and PowerShell/Python utility scripts used to operate the platform
Write and update unit and integration tests for transform logic
Participate in code reviews using Git feature branches and GitLab merge requests
Keep documentation current when system behavior changes
Build and maintain Databricks data pipelines for a financial services/wealth management data platform
Help ingest, transform, and publish data through bronze, silver, and gold layers, file exports, and Kafka
Work alongside a senior engineer and help keep environments healthy across dev/qa/uat
In the first 90 days, independently handle small transform bugs or enhancements, test changes, open merge requests, run operational scripts, query tables, diagnose failures, and begin taking ownership of Databricks environments

Requirements

What you’ll need

Solid Python fundamentals; ability to write clean, readable code using functions, modules, and basic OOP
Some exposure to Apache Spark / PySpark, or strong SQL skills plus willingness to learn Spark quickly
Working knowledge of SQL, including joins, aggregations, and window functions
Basic Git workflow knowledge: branches, commits, and pull/merge requests
Ability to read other people's code and stack traces and debug methodically
Direct Databricks experience is nice to have
Familiarity with Delta Lake, medallion architecture, or CDC/SCD concepts is nice to have
Azure exposure is nice to have
Kafka or other streaming/event systems experience is nice to have
CI/CD pipeline experience is nice to have
Prior financial services or regulated data environment experience is nice to have

Benefits

Comp & perks

Inclusive employer committed to diversity
Accommodation provided during the recruitment and selection process
Opportunity to grow fast and collaborate with exceptional teammates
Exposure to high-impact projects for scaling startups and global enterprises
Opportunity to learn and grow into a data engineering specialist