Senior Data Engineer

Penn Foster

. Architect, build, and evolve Penn Foster Group's modern cloud data platform and AI strategy .

Posted 11/3/2026full-timeRemote • United StatesSeniorWebsite

Core Competencies

Role fit

Core Competencies

Use this summary to align your resume positioning with the role.

Demonstrates expertise in Data Engineering and Analytics Engineering, with a strong focus on building and optimizing cloud data platforms using Databricks, SQL, and Python. Proven ability to lead technical initiatives, mentor teams, and implement data governance and quality practices.

Highest-signal resume keywords

Databricks Lakehouse Platform ExpertiseSQL and Python ProficiencyData Governance and Metadata ManagementCI/CD Pipeline ImplementationTechnical Leadership and Mentoring

ATS Keywords

Tailor your resume

Applicant Tracking System Keywords

Tip: use these terms in your resume and cover letter to boost ATS matches.

Hard Skills

Data EngineeringAnalytics EngineeringApache SparkDelta LakeSQLPythonMedallion ArchitectureData QualityObservabilityCloud Cost Optimization

Soft Skills

Strong CommunicationCollaboration SkillsInfluencing Stakeholders

Tools & Technologies

DatabricksPower BITableauMicrosoft AzureGitMLOpsAuto LoaderDelta Live TablesGenie SpacesGreat Expectations

Certifications & Qualifications

Databricks Certified Data Engineer ProfessionalMicrosoft Azure Certifications

Industry Keywords

Cloud Data PlatformData PipelinesBusiness IntelligenceData GovernanceMachine LearningGenerative AIOperational ExcellenceTechnical RoadmapsData ProductsSemantic Modeling

Tech Stack

Tools & technologies

ApacheAzureCloudPySparkPythonSparkSQLTableauUnity

About the role

Key responsibilities & impact

Architect, build, and evolve Penn Foster Group's modern cloud data platform and AI strategy
Provide technical leadership through architecture guidance, code reviews, mentoring, and engineering best practices
Mentor and develop junior and mid-level Data Engineers
Establish standards for software development, testing, CI/CD, documentation, observability, and operational excellence
Design, build, and support scalable enterprise data pipelines using SQL, Python, Apache Spark, and Databricks
Define Databricks development practices covering Workflows, Repos, Jobs, notebooks, Python libraries, Git integration, cluster policies, SQL Warehouses, and deployment automation
Design and optimize Delta Lake architectures using Medallion patterns, Delta optimization, Liquid Clustering, and Photon
Build reusable ingestion frameworks for batch, streaming, CDC, and API-based integrations
Optimize Spark workloads for performance, scalability, reliability, and cloud cost efficiency
Develop trusted semantic data products, Genie Spaces, and Genie Ontologies for Databricks Genie
Establish practices for semantic modeling, governed metrics, business metadata, and AI-ready datasets
Collaborate with Data Science and MLOps teams on machine learning, generative AI, and advanced analytics initiatives
Design scalable Lakehouse architectures for analytics, reporting, AI, and operational data products
Implement governance with Unity Catalog, including lineage, fine-grained security, metadata management, and access controls
Champion automated testing, monitoring, observability, data quality, and production reliability
Serve as the technical escalation point for complex production issues and root cause analysis
Drive platform modernization and contribute to technical roadmaps and platform strategy
Collaborate with Business Intelligence, Product, Platform Engineering, Security, Data Governance, and MLOps teams
Translate business requirements into technical solutions supporting reporting, analytics, AI, and strategic decision-making

Requirements

What you’ll need

Bachelor's degree in Computer Science, Engineering, Information Systems, or a related field, or equivalent practical experience
6+ years of experience in Data Engineering, Analytics Engineering, or Data Platform Engineering
2+ years of experience leading technical initiatives and mentoring engineering teams
Demonstrated success developing engineers while driving engineering excellence and technical standards
Strong communication and collaboration skills with the ability to influence technical and business stakeholders
Expert knowledge of the Databricks Lakehouse Platform, including Apache Spark (PySpark), Delta Lake, Unity Catalog, Databricks Workflows & Jobs, Repos, SQL Warehouses, Cluster Policies, Serverless Compute, MLflow, Lakehouse Monitoring, Auto Loader, and Delta Live Tables / Lakeflow
Expert knowledge of SQL and Python
Strong experience with BI tools such as Power BI, Tableau, or Business Objects
Strong Microsoft Azure experience, including ADLS Gen2, Microsoft Entra ID (Azure AD), RBAC, networking, and cloud security
Experience implementing Medallion Architecture, dimensional modeling, and domain-oriented data products
Deep understanding of Spark optimization techniques, including Adaptive Query Execution, partitioning, caching, Photon, Liquid Clustering, and Delta optimization
Experience implementing CI/CD pipelines, Git-based development workflows, Infrastructure as Code, and automated testing
Strong understanding of data governance, metadata management, data quality, observability, security, and compliance
Must reside and perform work within the United States
Must be willing and able to travel as needed
Must complete applicable pre-employment screening requirements
Must complete federal employment eligibility verification through Form I-9
Preferred: experience implementing Databricks Genie, Genie Spaces, and Genie Ontologies; semantic models and AI-ready data products; enterprise BI platforms; machine learning platforms, MLOps, or Generative AI applications; dbt, Great Expectations, or similar tools; Databricks Certified Data Engineer Professional and/or Microsoft Azure certifications

Benefits

Comp & perks

Medical insurance
Dental insurance
Vision insurance
Flexible spending accounts
Generous paid time off
Sponsored volunteer opportunities
401(k) with a company match
Free access to Penn Foster Group's online programs
Remote work arrangement
Sponsored travel to team meetings, company events, conferences, and summits as applicable