FREE ACCESS
5,000–10,000 jobs/day
See all jobs on Scoutfield
Search thousands of fresh jobs every day.
Discover
- Fresh listings
- Fast filters
- No subscription required
Create a free account and start exploring right away.
Core Competencies
Role fitCore Competencies
Use this summary to align your resume positioning with the role.
Demonstrates expertise in designing and operating large-scale data pipelines using Java, Python, and Spark, while ensuring data quality, observability, and governance. Proficient in integrating AI capabilities and leading technical direction in data engineering practices.
Highest-signal resume keywords
Java ProficiencyPython ProficiencySpark ExpertiseData Pipeline DesignMLOps Practices
ATS Keywords
Tailor your resumeApplicant Tracking System Keywords
Tip: use these terms in your resume and cover letter to boost ATS matches.
Hard Skills
Data EngineeringDistributed SystemsETL/ELT Pipeline DesignData GovernanceData QualitySystem DesignDomain-Oriented DesignBatch ProcessingStreaming ProcessingData Transformation
Soft Skills
Ownership MindsetTechnical JudgmentCollaborationMentoring
Tools & Technologies
Hadoop/HDFSHiveKafkaAirflowKubernetesCI/CDCloud-Native PracticesGenAI/LLM Tooling
Certifications & Qualifications
Bachelor's Degree in Computer ScienceMaster's Degree in Computer Science
Industry Keywords
Big DataData Quality MetricsObservabilityIncident ManagementBlameless Post-Incident Reviews
Tech Stack
Tools & technologiesAirflowCloudDistributed SystemsETLHadoopHDFSJavaKafkaKubernetesMicroservicesPythonSpark
About the role
Key responsibilities & impact- Design, build, and operate large-scale batch and streaming data pipelines using Spark, Java, and Python
- Apply software engineering fundamentals, including clean architecture, automated testing, CI/CD, and code review
- Apply domain-oriented design, system design, and low-level design to data and platform problems
- Build and maintain services, APIs, and distributed systems supporting data pipelines
- Integrate AI, GenAI, and agentic capabilities into the data platform
- Partner with data scientists and ML engineers on feature pipelines, model-serving data paths, and MLOps infrastructure
- Own data quality, observability, and governance for pipelines and datasets
- Participate in and lead architecture and design reviews
- Set technical direction and best practices, mentor engineers, and influence data engineering standards
- Evaluate and adopt emerging Big Data, cloud, and AI/agentic technologies
Requirements
What you’ll need- 8+ years of professional software engineering experience with significant hands-on data engineering at scale
- Strong proficiency in Java and Python
- Strong working knowledge of Spark for distributed data processing
- Operational Excellence and Engineering Excellence skills, including SLO/SLA ownership, incident management, blameless post-incident reviews, reliability and quality metrics, and continuous improvement
- Bachelor's or Master's degree in Computer Science, Engineering, or a related technical field, or equivalent practical experience
- Deep experience with Spark, Hadoop/HDFS, Hive, Kafka, workflow orchestration such as Airflow, batch and streaming processing, and large-scale data transformation
- Experience with data modeling, ETL/ELT pipeline design, data quality and observability, data governance, and lakehouse architectures
- Hands-on experience integrating model APIs and GenAI/LLM tooling, building or operating agentic workflows, embeddings/vector search, and MLOps practices
- Working literacy in microservices, API design, Kubernetes or equivalent orchestration, CI/CD, and cloud-native practices
- Strong distributed systems, system design, domain-oriented design, and low-level design skills
- Strong software engineering craftsmanship, including testing strategy, code review standards, and design patterns
- Ability to translate ambiguous data/business problems into technical execution plans
- Strong ownership mindset and technical judgment
- Minimum qualification alternatives: Bachelor's degree in Computer Science and 4 years' experience in software engineering or related field; or 6 years' experience in software engineering or related field; or Master's degree in Computer Science and 2 years' experience in software engineering or related field
- 3 years' experience in data engineering, database engineering, business intelligence, or business analytics
Benefits
Comp & perks- Incentive awards for performance
- Maternity and parental leave
- PTO
- Health benefits
- Flexibility for associates to manage their personal lives