FREE ACCESS
5,000–10,000 jobs/day
See all jobs on Scoutfield
Search thousands of fresh jobs every day.
Discover
- Fresh listings
- Fast filters
- No subscription required
Create a free account and start exploring right away.
Core Competencies
Role fitCore Competencies
Use this summary to align your resume positioning with the role.
Demonstrates extensive experience in designing and operating large-scale data pipelines using Java, Python, and Spark, with a strong focus on data quality, observability, and governance. Proficient in applying software engineering principles and best practices to enhance data infrastructure and support AI and analytics initiatives.
Highest-signal resume keywords
Java ProficiencyPython ProficiencySpark ExpertiseData Quality GovernanceMLOps Practices
ATS Keywords
Tailor your resumeApplicant Tracking System Keywords
Tip: use these terms in your resume and cover letter to boost ATS matches.
Hard Skills
Data EngineeringDistributed SystemsETL/ELT Pipeline DesignData ModelingBatch ProcessingStreaming ProcessingWorkflow OrchestrationSystem DesignLow-Level DesignSoftware Engineering Craftsmanship
Soft Skills
OwnershipTechnical JudgmentAutonomous OperationMentoringCollaboration
Tools & Technologies
HadoopHDFSHiveKafkaAirflowDelta LakeBigQueryDatabricksKubernetesCI/CD
Certifications & Qualifications
Bachelor's Degree in Computer ScienceMaster's Degree in Computer Science
Industry Keywords
Big DataCloud TechnologiesAIGenAIData GovernanceObservabilityData TransformationLakehouse ArchitecturesMicroservicesAPI Design
Tech Stack
Tools & technologiesAirflowBigQueryCloudDistributed SystemsETLHadoopHDFSJavaKafkaKubernetesMicroservicesPythonSpark
About the role
Key responsibilities & impact- Design, build, and operate large-scale batch and streaming data pipelines using Spark, Java, and Python
- Apply software engineering fundamentals, including clean architecture, automated testing, CI/CD, code review, and design discipline
- Apply domain-oriented design, system design, and low-level design to data and platform problems
- Build and maintain services, APIs, and distributed systems supporting data pipelines
- Integrate AI, GenAI, embeddings/vector search, LLM-assisted data quality, anomaly detection, agentic monitoring, self-healing, schema, and lineage documentation into the data platform
- Partner with data scientists and ML engineers on feature pipelines, model-serving data paths, and MLOps infrastructure
- Own data quality, observability, and governance for pipelines and datasets
- Participate in and lead architecture and design reviews across data and platform engineering
- Set technical direction and best practices across teams; mentor engineers and influence data engineering standards
- Evaluate and adopt Big Data, cloud, and AI/agentic technologies
- Work within Walmart's GDF platform team building foundational data infrastructure for AI, analytics, and decision-making across Walmart
Requirements
What you’ll need- 12+ years of professional software engineering experience with significant hands-on data engineering at scale
- Strong proficiency in Java and Python
- Strong working knowledge of Spark for distributed data processing
- Operational Excellence and Engineering Excellence skills, including SLO/SLA ownership, incident management, blameless post-incident reviews, reliability and quality metrics, and continuous improvement
- Bachelor's or Master's degree in Computer Science, Engineering, or a related technical field, or equivalent practical experience
- Deep experience with Spark, Hadoop/HDFS, Hive, Kafka, workflow orchestration such as Airflow, batch and streaming processing, and large-scale data transformation
- Experience with data modeling, ETL/ELT pipeline design, data quality and observability, data governance, and lakehouse architectures such as Delta Lake, BigQuery, and Databricks
- Hands-on experience with model APIs, GenAI/LLM tooling, agentic workflows, embeddings/vector search, and MLOps practices
- Working literacy in microservices, API design, Kubernetes or equivalent orchestration, CI/CD, and cloud-native practices
- Strong distributed systems, system design, domain-oriented design, and low-level design skills
- Strong software engineering craftsmanship, including testing strategy, code review standards, and design patterns
- Ability to translate ambiguous data/business problems into technical execution plans
- Strong ownership, technical judgment, and ability to operate autonomously
- Minimum qualification alternatives: Bachelor's degree in Computer Science and 5 years' software engineering or related experience; or 7 years' software engineering or related experience; or Master's degree in Computer Science and 3 years' software engineering or related experience
- 4 years' experience in data engineering, database engineering, business intelligence, or business analytics
Benefits
Comp & perks- Incentive awards for performance
- Maternity and parental leave
- PTO
- Health benefits
- Flexibility for associates to manage their personal lives