FREE ACCESS
5,000–10,000 jobs/day
See all jobs on Scoutfield
Search thousands of fresh jobs every day.
Discover
- Fresh listings
- Fast filters
- No subscription required
Create a free account and start exploring right away.
Core Competencies
Role fitCore Competencies
Use this summary to align your resume positioning with the role.
Demonstrates expertise in designing and operating data pipelines on cloud platforms, with a strong focus on ETL/ELT processes using SQL, Python, and PySpark. Proficient in implementing data governance practices and CI/CD pipelines while ensuring data quality and compliance.
Highest-signal resume keywords
SQLPythonPySparkData GovernanceCI/CD Practices
ATS Keywords
Tailor your resumeApplicant Tracking System Keywords
Tip: use these terms in your resume and cover letter to boost ATS matches.
Hard Skills
ETL ProcessesELT ProcessesData ArchitectureData ModelingData Quality ChecksAnomaly DetectionInfrastructure-as-CodeTerraformBicepCloud Data Platforms
Soft Skills
Engineering JudgmentCollaborationProblem-Solving
Tools & Technologies
Microsoft PurviewGitAzureAWSDatabricks
Industry Keywords
Data EngineeringData Platform EngineeringAI GovernanceMetadata EnrichmentAccess Control
Tech Stack
Tools & technologiesAWSAzureCloudETLPySparkPythonSQLTerraform
About the role
Key responsibilities & impact- Design, build, and operate batch and streaming data pipelines on modern cloud data platforms
- Develop robust ETL/ELT processes using SQL, Python, and PySpark with error handling, monitoring, and cost awareness
- Implement layered/medallion data architectures and analytics-ready data models for BI and AI workloads
- Partner with analysts and data scientists to deliver trusted, production-grade data assets
- Support and enhance data governance practices using Microsoft Purview or comparable platforms
- Contribute to governance standards for data products, analytics solutions, AI/ML features, and agent-based workflows
- Incorporate governance into platform workflows and data engineering practices
- Contribute to AI-enabled tools and automation for metadata enrichment, data quality checks, anomaly detection, and governance workflows
- Ensure AI-enabled tools operate only on approved, governed data sources with logging and auditability
- Build knowledge of AI governance patterns, agent lifecycle management, and input/output traceability
- Implement and maintain CI/CD pipelines for data and AI assets
- Promote infrastructure-as-code practices using Terraform, Bicep, or equivalent
- Define development, test, and production environment promotion paths with governance and policy checks
Requirements
What you’ll need- Bachelor’s degree AND 5+ years of experience in data engineering or data platform engineering in a production environment
- SQL and hands-on experience with Python and PySpark
- Experience in data engineering on at least one major cloud or data platform (Azure, AWS, Databricks, or equivalent)
- Transferable skills across modern cloud and data platforms
- Experience with Git-based development and CI/CD practices
- Working knowledge of data governance concepts such as cataloging, lineage, data quality, or access control
- Demonstrated engineering judgment and ability to balance delivery speed, reliability, safety, and governance considerations
- Direct submission of the application by the applicant
- Ability to comply with the company’s prohibition on AI tools during interviews, unless an accommodation is arranged
Benefits
Comp & perks- Equal opportunity employer
- Reasonable accommodations for disabled veterans, individuals with disabilities, and individuals with sincerely held religious beliefs
- Confidential accommodation support during application and employment
- Benefits offerings referenced under the Transparency in (Benefits) Coverage Act