FREE ACCESS
5,000–10,000 jobs/day
See all jobs on Scoutfield
Search thousands of fresh jobs every day.
Discover
- Fresh listings
- Fast filters
- No subscription required
Create a free account and start exploring right away.
Core Competencies
Role fitCore Competencies
Use this summary to align your resume positioning with the role.
Demonstrates expertise in designing and maintaining ETL/ELT pipelines using Spark and Scala on AWS, with a strong focus on performance tuning and data quality management. Proficient in leveraging generative AI tools for data engineering tasks and possesses solid experience in big data ecosystems.
Highest-signal resume keywords
Spark Application TuningETL/ELT Pipeline DevelopmentAWS EMR and S3Data Quality ManagementGenerative AI in Data Engineering
ATS Keywords
Tailor your resumeApplicant Tracking System Keywords
Tip: use these terms in your resume and cover letter to boost ATS matches.
Hard Skills
SparkScalaDataFrame APIDataset APIRelational Database ManagementBig Data DebuggingData StructuresData ModelingProbabilistic Data StructuresHigh Cardinality Data Systems
Soft Skills
Problem-SolvingCollaborationCommunication
Tools & Technologies
AWS EMRSnowflakeGrafanaKafkaHadoopElastic StackDocker
Certifications & Qualifications
Bachelor's Degree in Computer ScienceComputer Engineering
Industry Keywords
ETLELTBig DataData EngineeringSoftware Development Lifecycle
Tech Stack
Tools & technologiesAWSDockerETLGrafanaHadoopKafkaScalaSnowflakeSpark
About the role
Key responsibilities & impact- Design, develop, and maintain ETL/ELT pipelines powering the Resonate business using Spark and Scala on AWS EMR
- Work with terabyte-scale datasets across S3 and Snowflake
- Tune multi-terabyte and petabyte-scale Spark applications for performance and cost
- Debug production-scale big data problems
- Partner with senior engineers and product management to take product requests from design and planning through production
- Monitor pipeline health in Grafana
- Track down and resolve data quality issues at their source
- Write clean, testable code with thorough unit and integration tests
- Use Gen AI tools across pipeline development, monitoring, and daily engineering work
- Participate in code reviews, technical design discussions, and sprint planning
- Contribute to architecture decisions
- Support production operations, including on-call rotation and incident response
Requirements
What you’ll need- Around five years or more of professional experience in software engineering, data engineering, or a closely related field
- At least three years hands-on with Spark and Scala, specifically the DataFrame and Dataset APIs
- Proven experience tuning Spark applications at multi-terabyte or petabyte scale
- Real debugging and problem-solving experience in the big data ecosystem, including production failures
- Solid relational database experience
- Working knowledge of AWS (EMR, S3, Lambda), Kafka, Snowflake, Grafana, Hadoop, Elastic Stack, and Docker
- Strong grasp of the full software development lifecycle, from exploration and design through production delivery
- Good instincts on solution architecture, data structures, and data modeling
- Enthusiasm for using generative AI to accelerate data engineering work
- Hands-on experience building and maintaining big data pipelines with Gen AI
- Experience using probabilistic data structures
- Track record of implementing high cardinality data systems at scale
- Bachelor's degree in computer science, computer engineering, or equivalent experience is a strong signal
Benefits
Comp & perks- Annual bonus program tied to company performance
- Medical, dental, and vision coverage, including HSA
- Company-paid disability and life insurance with a life insurance buy-up option
- 401(k) with company match
- Flexible PTO plus 10 paid holidays
- Paid parental leave
- Access to wellbeing and employee assistance programs, including free and unlimited access to 20+ health professionals and wellbeing coaches for you and your family members
- Remote-first environment with genuine flexibility
- Occasional travel for company and client events
