FREE ACCESS
5,000–10,000 jobs/day
See all jobs on Scoutfield
Search thousands of fresh jobs every day.
Discover
- Fresh listings
- Fast filters
- No subscription required
Create a free account and start exploring right away.
Core Competencies
Role fitCore Competencies
Use this summary to align your resume positioning with the role.
Demonstrates expertise in designing and optimizing large-scale data pipelines and distributed data systems, with a strong focus on data governance, quality, and operational excellence. Proficient in leading data engineering initiatives and mentoring teams to deliver scalable solutions that support analytics and machine learning.
Highest-signal resume keywords
Big Data EngineeringHadoopApache SparkETL ProcessesData Pipeline Management
ATS Keywords
Tailor your resumeApplicant Tracking System Keywords
Tip: use these terms in your resume and cover letter to boost ATS matches.
Hard Skills
PythonSQLData ModelingMachine LearningData GovernanceContainerizationUnix/Shell ScriptingAirflowControl-MAgile Development
Soft Skills
LeadershipMentoringCommunicationTeam CollaborationStrong Work Ethic
Tools & Technologies
DockerKubernetesKafkaRedisFlinkTensorFlowTritonAWS Services
Industry Keywords
Data EngineeringData QualityOperational ExcellenceReal-Time ProcessingBatch Processing
Tech Stack
Tools & technologiesAirflowApacheAWSDockerETLHadoopKafkaKubernetesPythonRedisSparkSQLTensorflowUnix
About the role
Key responsibilities & impact- Design, build, and optimize large-scale data pipelines for batch and real-time processing
- Develop and maintain highly available, scalable, and resilient distributed data systems
- Lead initiatives to process, transform, and manage structured and unstructured data across platforms
- Drive development of data platforms and products supporting analytics, reporting, and machine learning use cases
- Partner with product, analytics, and business teams to translate requirements into scalable data solutions
- Architect solutions using Hadoop, Spark, and distributed data processing frameworks
- Build and optimize ETL/ELT pipelines while ensuring data quality, reliability, and performance
- Implement automation, monitoring, and observability across the data lifecycle
- Lead proof-of-concept initiatives for new data engineering technologies
- Influence data engineering best practices, including data modeling, pipeline design, and governance
- Mentor engineers and foster continuous learning and accountability
- Collaborate with stakeholders to understand data requirements and deliver impactful solutions
- Architect and implement secure, robust, and scalable data platforms
- Drive innovation through experimentation, prototyping, and rapid iteration
- Ensure strong data governance, quality, and operational excellence
- Engage cross-functional teams to align technical and business outcomes
Requirements
What you’ll need- 5 or more years of relevant work experience with a Bachelors Degree, or at least 2 years of work experience with an Advanced degree, or 0 years of work experience with a PhD
- Preferred: 6 or more years of work experience with a Bachelors Degree, or 4 or more years of relevant experience with an Advanced Degree, or up to 3 years of relevant experience with a PhD
- Demonstrated leadership in delivering high-quality, large-scale, enterprise-class applications
- Solid experience in big data engineering, including Hadoop, Apache Spark, Python, and SQL
- Experience with Docker and Kubernetes for containerization
- Proficiency in creating and managing large-scale data pipelines and machine learning models
- Experience developing ETL processes, maintaining Spark pipelines, and productizing AI/ML models
- Proficiency with Kafka, Redis, Flink, TensorFlow, Triton, and AWS services
- Unix/Shell or Python scripting skills and experience with Airflow and Control-M
- Familiarity with Agile development, TDD, CI/CD, and various databases
- Proven track record of building reliable, scalable, and operable applications
- Ability to manage component security analysis and collaborate with security teams
- Strong work ethic, focus on immediate goals, and experience as a technical leader
- Passion for mentoring and helping juniors grow professionally
- Excellent communication and interpersonal skills, and a strong team player
- Ability to travel 5–10% of the time
Benefits
Comp & perks- Potential bonus and equity eligibility
- Medical insurance
- Dental insurance
- Vision insurance
- 401(k)
- FSA/HSA
- Life insurance
- Paid Time Off
- Wellness Program