FREE ACCESS
5,000–10,000 jobs/day
See all jobs on Scoutfield
Search thousands of fresh jobs every day.
Discover
- Fresh listings
- Fast filters
- No subscription required
Create a free account and start exploring right away.
Core Competencies
Role fitCore Competencies
Use this summary to align your resume positioning with the role.
Demonstrates expertise in designing and operating large-scale data pipelines using Spark, Java, and Python, while ensuring data quality and governance. Proficient in integrating AI capabilities and collaborating with data scientists on MLOps infrastructure.
Highest-signal resume keywords
Java ProficiencyPython ProficiencySpark ExpertiseData Engineering ExperienceMLOps Practices
ATS Keywords
Tailor your resumeApplicant Tracking System Keywords
Tip: use these terms in your resume and cover letter to boost ATS matches.
Hard Skills
Data Pipeline DesignDistributed SystemsETL/ELT Pipeline DesignData Quality and ObservabilityData GovernanceBatch and Streaming ProcessingData ModelingWorkflow OrchestrationMicroservicesSystem Design
Soft Skills
Ownership MindsetTechnical JudgmentMentoring
Tools & Technologies
Hadoop/HDFSHiveKafkaAirflowDelta LakeBigQueryDatabricksKubernetesCI/CDGenAI/LLM Tooling
Certifications & Qualifications
Bachelor's Degree in Computer ScienceMaster's Degree in Computer Science
Industry Keywords
Big DataCloud TechnologiesData Engineering StandardsDomain-Oriented DesignClean Architecture
Tech Stack
Tools & technologiesAirflowBigQueryCloudDistributed SystemsETLHadoopHDFSJavaKafkaKubernetesMicroservicesPythonScalaSparkC++Go
About the role
Key responsibilities & impact- Design, build, and operate large-scale batch and streaming data pipelines using Spark, Java, and Python
- Apply software engineering fundamentals, including clean architecture, automated testing, CI/CD, and code review discipline
- Apply domain-oriented design, system design, and low-level design rigor to data and platform problems
- Build and maintain services, APIs, and distributed systems supporting data pipelines
- Integrate AI, GenAI, and agentic capabilities into the data platform
- Partner with data scientists and ML engineers on feature pipelines, model-serving data paths, and MLOps infrastructure
- Own data quality, observability, and governance for pipelines and datasets
- Participate in and lead architecture and design reviews
- Set technical direction and best practices, mentor engineers, and influence data engineering standards
- Evaluate and adopt emerging Big Data, cloud, and AI/agentic technologies
Requirements
What you’ll need- 8+ years of professional software engineering experience with significant hands-on data engineering at scale
- Strong proficiency in Java and Python
- Strong working knowledge of Spark for distributed data processing
- Bachelor's or Master's degree in Computer Science, Engineering, or a related technical field, or equivalent practical experience
- Minimum qualification alternatives: Bachelor's degree in Computer Science and 4 years' experience in software engineering or related field; 6 years' experience in software engineering or related field; or Master's degree in Computer Science and 2 years' experience in software engineering or related field
- 3 years' experience in data engineering, database engineering, business intelligence, or business analytics
- Deep experience with Spark, Hadoop/HDFS, Hive, Kafka, workflow orchestration such as Airflow, batch and streaming processing, and large-scale data transformation
- Experience with data modeling, ETL/ELT pipeline design, data quality and observability, data governance, and lakehouse architectures such as Delta Lake, BigQuery, and Databricks
- Hands-on experience integrating model APIs and GenAI/LLM tooling, building or operating agentic workflows, embeddings/vector search, and MLOps practices
- Working literacy in microservices, API design, Kubernetes or equivalent orchestration, CI/CD, and cloud-native practices
- Strong distributed systems, system design, domain-oriented design, and low-level design skills
- Strong software engineering craftsmanship, including testing strategy, code review standards, and design patterns
- Ability to translate ambiguous data/business problems into technical execution plans
- Strong ownership mindset and technical judgment
- Additional experience with Go, C++, or Scala is a plus
Benefits
Comp & perks- Incentive awards for performance
- Maternity and parental leave
- PTO
- Health benefits
- Flexibility for associates to manage their personal lives
