Senior Staff Engineer, Data Platform

Qualified Health

. Set the technical direction and architecture for the Data Platform across multiple teams .

Posted 11/12/2026full-timeRemote • United StatesSenior💰 $215,000 - $235,000 per yearWebsite

Core Competencies

Role fit

Core Competencies

Use this summary to align your resume positioning with the role.

Demonstrates expertise in architecting and optimizing data platforms, particularly with Databricks, Spark, and SQL, while ensuring compliance and governance in regulated domains. Proven ability to mentor teams, communicate effectively with stakeholders, and drive engineering effectiveness through strategic decision-making.

Highest-signal resume keywords

Databricks ArchitectureSpark Pipeline OptimizationData Governance and CompliancePython and SQL FluencyHealthcare Data Experience

ATS Keywords

Tailor your resume

Applicant Tracking System Keywords

Tip: use these terms in your resume and cover letter to boost ATS matches.

Hard Skills

Data Platform ArchitectureSparkSQLDelta LakeUnity CatalogML Pipeline DevelopmentAgentic Coding ToolsCI/CD PracticesInfrastructure as CodePerformance Tuning

Soft Skills

Clear CommunicationMentoringInfluencing Without AuthorityProblem-SolvingTeam Collaboration

Tools & Technologies

DatabricksAirflowGCPAWSAzureTerraformKubernetesClaude CodeDelta Live TablesLakeflow

Certifications & Qualifications

Databricks Certifications

Industry Keywords

HealthcareInsuranceFinancePHI ComplianceEHRFHIRHL7HIPAA

Tech Stack

Tools & technologies

AirflowAWSAzureGoogle Cloud PlatformKubernetesPythonSnowflakeSparkSQLTerraformUnity

About the role

Key responsibilities & impact

Set the technical direction and architecture for the Data Platform across multiple teams
Architect and evolve the Databricks lakehouse, including medallion layering, Delta Lake, Unity Catalog governance, and workspace and compute strategy
Design, build, and optimize Spark and SQL pipelines on Databricks
Define data contracts, quality checks, access controls, and PHI-aware governance with security and compliance
Build data foundations for AI products, including LLM-ready datasets, retrieval and vector-store pipelines, evaluation data, and governed interfaces
Design durable Temporal workflows for AI pipelines, including scheduled and watermarked pulls, file ingestion, and long-running LLM extraction and enrichment jobs
Establish retry, idempotency, versioning, backfill, observability, and governance patterns across Databricks and Temporal
Use AI coding agents daily and define team practices for agentic development
Write and review technical designs, set platform standards, and make long-term build-versus-buy decisions
Mentor engineers, unblock teams, and communicate trade-offs to technical and non-technical stakeholders
Shape the roadmap with leadership and measurably improve engineering effectiveness while building complex components hands-on

Requirements

What you’ll need

Bachelor's Degree in Computer Science, Electrical Engineering, Mathematics or a related field
At least 12+ years of software or data engineering experience
Track record of designing and operating production data platforms at scale and setting architecture across multiple teams
Experience working in a highly regulated domain (Healthcare, Insurance, Finance) required
Deep hands-on expertise with Databricks, or aligned expertise in Snowflake or Microsoft Fabric
Experience with Spark, Delta Lake, Unity Catalog, Workflows or equivalent, and performance and cost tuning at scale
Experience building pipelines for ML or LLM applications
Fluency in Python and SQL
Software engineering practices including testing, CI/CD, infrastructure as code, and code review
Demonstrated practical use of agentic coding tools such as Claude Code or similar
Experience owning ambiguous, cross-team problems end to end and influencing without direct authority
Clear written and verbal communication, including authoring design documents
Healthcare data experience with claims, EHR, FHIR or HL7, and HIPAA-compliant handling of PHI desirable
Databricks certifications or experience with Delta Live Tables, Lakeflow, or Databricks Asset Bundles desirable
Experience with Airflow, Dagster, Prefect, GCP, AWS, Azure, Terraform, or Kubernetes desirable
Contributions to open-source data, workflow, or AI infrastructure projects desirable
Legally authorized to work in the United States for any employer without requiring sponsorship now or in the future

Benefits

Comp & perks

Equity packages
Medical insurance
Dental insurance
Vision insurance
Flexible working hours
Hybrid work options
Inclusive environment that fosters creativity and innovation