FREE ACCESS
5,000–10,000 jobs/day
See all jobs on Scoutfield
Search thousands of fresh jobs every day.
Discover
- Fresh listings
- Fast filters
- No subscription required
Create a free account and start exploring right away.
Core Competencies
Role fitCore Competencies
Use this summary to align your resume positioning with the role.
Proficient in developing ETL/ELT pipelines using Databricks, PySpark, and Delta Lake, with a strong focus on data quality practices and performance optimization. Experienced in collaborating with cross-functional teams to deliver robust data solutions on AWS, leveraging tools like Amazon Redshift and AWS Glue.
Highest-signal resume keywords
ETL/ELT Pipeline DevelopmentAWS Engineering ExperiencePython ProgrammingData Quality PracticesSpark Performance Tuning
ATS Keywords
Tailor your resumeApplicant Tracking System Keywords
Tip: use these terms in your resume and cover letter to boost ATS matches.
Hard Skills
DatabricksPySparkSpark SQLDelta LakeSQLData Pipeline DevelopmentData Quality ChecksIncremental LoadingDimensional Data ModelingSchema Validation
Soft Skills
CollaborationTroubleshootingCommunication
Tools & Technologies
Amazon S3Amazon RedshiftPower BITableauAmazon QuickSightApache AirflowAWS GlueGitCI/CD Workflows
Industry Keywords
Medallion ArchitectureCloud Resource OptimizationAgile DeliveryData Layer Management
Tech Stack
Tools & technologiesAirflowAmazon RedshiftApacheAWSCloudETLOraclePySparkPythonSparkSQLTableau
About the role
Key responsibilities & impact- Develop batch and incremental ETL/ELT pipelines using Databricks, PySpark, Spark SQL, and Delta Lake
- Build and maintain Bronze, Silver, and Gold data layers using Medallion Architecture
- Ingest and transform structured and semi-structured data from Oracle, SQL Server, REST APIs, and Amazon S3
- Implement data quality checks, including schema validation, deduplication, reconciliation, and business rule enforcement
- Optimize Spark jobs and Delta tables to improve processing time and control cloud resource use
- Publish curated datasets to Amazon Redshift and support reporting in Power BI, Tableau, or Amazon QuickSight
- Orchestrate, monitor, and troubleshoot production pipelines using Databricks Workflows, Apache Airflow, and AWS monitoring services
- Collaborate with analysts, architects, QA, and DevOps teams on requirements, releases, and production support
Requirements
What you’ll need- 5+ years of overall technology experience
- 3+ years of AWS engineering experience
- Strong Python and SQL skills
- Hands-on experience with PySpark, Databricks, and Delta Lake
- Experience building data pipelines on AWS, particularly with Amazon S3, AWS Glue, and Amazon Redshift
- Knowledge of incremental loading, dimensional data modeling, data quality practices, and Spark performance tuning
- Experience with Git, CI/CD workflows, and Agile delivery
