FREE ACCESS
5,000–10,000 jobs/day
See all jobs on Scoutfield
Search thousands of fresh jobs every day.
Discover
- Fresh listings
- Fast filters
- No subscription required
Create a free account and start exploring right away.
Core Competencies
Role fitCore Competencies
Use this summary to align your resume positioning with the role.
Demonstrates expertise in building and maintaining data pipelines and backend systems using Azure Databricks, with a strong focus on data modeling, quality, and observability. Proficient in implementing CI/CD practices and leveraging AI tooling for analytics.
Highest-signal resume keywords
Azure DatabricksApache SparkData ModelingCI/CD PracticesData Quality Implementation
ATS Keywords
Tailor your resumeApplicant Tracking System Keywords
Tip: use these terms in your resume and cover letter to boost ATS matches.
Hard Skills
Databricks SQLPySparkSpark SQLStructured StreamingDelta LakeSQL OptimizationPythonData Pipeline DevelopmentData TransformationData Governance
Soft Skills
Problem-SolvingCollaborationOwnershipCommunicationIncident Management
Tools & Technologies
Unity CatalogLakeflow JobsDelta Live TablesAnthropic ClaudeOpenAI APIsVersion ControlAutomated TestingMonitoring ToolsData Incident TriageEmbedded Analytics
Certifications & Qualifications
Databricks Certified Data Engineer AssociateDatabricks Certified Data Engineer Professional
Industry Keywords
Medallion Data ModelsDimensional Data ModelsMulti-Tenant SaaS DataSQL ServerAzure SQL Managed InstanceChange Data CaptureLakebasePostgreSQLAnalytics AgentData Contracts
Tech Stack
Tools & technologiesApacheAzurePostgresPySparkPythonSparkSQLUnity
About the role
Key responsibilities & impact- Build and run the lakehouse on Azure Databricks
- Develop Spark and Delta Lake transformations, orchestration, and serving layers
- Implement and maintain medallion and dimensional data models in Delta Lake
- Tune Spark jobs, Photon, liquid clustering, storage, compute, and DBU spend
- Maintain the governed Unity Catalog semantic layer and certified Metric Views
- Enforce per-tenant row and column security at query time
- Implement and reconcile StoreForce retail KPI logic
- Onboard tenants and add KPIs through configuration and established patterns
- Enforce data quality, data contracts, reconciliation, freshness, and volume checks
- Maintain lineage and Lakehouse Monitoring and handle data incidents as production incidents
- Deliver infrastructure and platform changes as code through CI/CD
- Build and operate an LLM-powered analytics agent using Databricks AI tooling, Anthropic Claude, and OpenAI
- Ensure tenant isolation, runtime guardrails, and automated evaluation quality gates
- Work with the implementation partner during initial build and take ownership at handover
- Serve as StoreForce’s go-to data engineer and support consuming teams
- Maintain engineering standards and runbooks and triage data incidents
Requirements
What you’ll need- 5+ years building data pipelines or backend systems, including at least 2 years running a modern data stack in production
- Hands-on production experience with Databricks: Unity Catalog, Delta Lake, Lakeflow Jobs and Spark Declarative Pipelines (or Workflows and Delta Live Tables), Databricks SQL, and Databricks Asset Bundles or equivalent as-code practice
- Strong Apache Spark skills: PySpark, Spark SQL, Structured Streaming, query-plan analysis, shuffle and skew diagnosis, and job and compute tuning
- Strong SQL, including window functions, complex joins, and query optimization
- Strong Python for data work
- Solid data modeling across medallion, dimensional, and incremental patterns
- Experience tuning Delta tables, including liquid clustering, change data feed, OPTIMIZE, and VACUUM strategy
- Experience implementing data quality and observability, including expectations, contracts, reconciliation, lineage, monitoring, and alerting
- Hands-on experience with Anthropic Claude and OpenAI APIs for AI features
- Fluent in modern software engineering practices, including version control, automated testing, and CI/CD
- Databricks Certified Data Engineer Associate or Professional preferred
- Experience with LLMs, multi-tenant SaaS data, SQL Server/Azure SQL Managed Instance CDC, Lakebase/PostgreSQL, embedded analytics, and platform handover preferred
- Candidates must be based in the United States or Canada
Benefits
Comp & perks- Flexible schedule
- Employer-paid health benefits including dental, prescription drug coverage, and a Health Spending Account
- RRSP Matching with First Time Home Buyer support
- Company paid training and professional development
- Extended Long Weekends and Religious Observance Days
- Employee Referral Program
- Spotlight Recognition Bonus Program
- Dedicated Employee Assistance Program (EAP)
- Accommodations available on request for candidates participating in all aspects of the selection process