FREE ACCESS
5,000–10,000 jobs/day
See all jobs on Scoutfield
Search thousands of fresh jobs every day.
Discover
- Fresh listings
- Fast filters
- No subscription required
Create a free account and start exploring right away.
Data Engineer II
American Express. Design, develop and maintain scalable data pipelines using GCP BigQuery, Python, SQL, Spark and Apache Airflow .
Core Competencies
Role fitCore Competencies
Use this summary to align your resume positioning with the role.
Demonstrates expertise in designing and maintaining scalable data pipelines using GCP BigQuery, Python, and SQL, while leveraging Generative AI tools for enhanced coding and automation. Strong analytical skills in data profiling, validation, and reconciliation support effective data-driven decision-making.
Highest-signal resume keywords
GCP BigQueryPythonSQLApache AirflowData Analysis
ATS Keywords
Tailor your resumeApplicant Tracking System Keywords
Tip: use these terms in your resume and cover letter to boost ATS matches.
Hard Skills
Data Pipeline DevelopmentETL/ELTData ModelingData TransformationWorkflow OrchestrationPerformance OptimizationComplex SQL AnalysisRoot Cause AnalysisData ValidationData Profiling
Soft Skills
Strong Communication SkillsCollaborationProblem-Solving
Tools & Technologies
GitHubCI/CDGenerative AI ToolsSparkHadoopKafkaAdobe Experience PlatformAdobe Journey OptimizerDataflowPub/Sub
Certifications & Qualifications
GCP Professional Data Engineer
Industry Keywords
Big DataCloud-Based SolutionsData GovernanceData Quality ControlsMarketing AnalyticsCustomer Engagement
Tech Stack
Tools & technologiesAirflowApacheBigQueryCloudETLGoogle Cloud PlatformHadoopKafkaPythonScalaSparkSQL
About the role
Key responsibilities & impact- Design, develop and maintain scalable data pipelines using GCP BigQuery, Python, SQL, Spark and Apache Airflow
- Perform data discovery, profiling, complex SQL analysis, source-to-target reconciliation, data validation and root cause analysis
- Leverage approved Generative AI tools, GitHub Copilot and LLM-based solutions for code development, testing, troubleshooting, documentation and engineering automation
- Optimize data-processing performance, automate monitoring, troubleshoot production issues and implement preventive measures
- Build reusable customer, product, campaign and engagement data capabilities supporting enterprise Marketing, targeting, personalization and analytics
- Follow software engineering best practices, including CI/CD, GitHub, code reviews, automated testing, data governance and production-readiness controls
- Partner with Product Owners, business stakeholders, architects and upstream/downstream engineering teams to understand requirements, resolve dependencies and deliver business capabilities
- Identify opportunities for automation, technical debt reduction, AI adoption and engineering productivity improvements
Requirements
What you’ll need- Bachelor's or master's degree in computer science, Engineering or a related technical discipline, or equivalent work experience
- 5+ years of hands-on experience in software development or data engineering, preferably with large-scale Big Data and cloud-based solutions
- Strong hands-on expertise in SQL, Python, GCP BigQuery and Apache Airflow
- Experience with Spark, Scala, Hadoop/Hive or equivalent distributed data-processing technologies
- Strong data analysis and problem-solving skills, including data profiling, complex SQL, source-to-target validation, data reconciliation and root cause analysis
- Experience in ETL/ELT, data modeling, data transformation, workflow orchestration and performance optimization
- Practical hands-on experience with Generative AI tools such as GitHub Copilot, ChatGPT Enterprise or equivalent AI coding assistants
- Working knowledge of LLMs, prompt engineering and responsible AI practices, particularly secure handling of enterprise and sensitive data
- Experience with Git/GitHub, CI/CD pipelines, automated testing and software development best practices
- Understanding of data-quality controls, data governance, privacy and production support
- Strong communication skills, ability to collaborate across teams and independently resolve complex technical problems
- GCP Bigtable, Pub/Sub, Dataflow, Kafka or real-time data-processing frameworks (preferred)
- Adobe Experience Platform, Adobe Journey Optimizer or Customer Journey Analytics (preferred)
- AI-enabled data engineering solutions, LLM APIs, RAG, agentic AI frameworks or AI workflow automation (preferred)
- Experience developing enterprise data products supporting Marketing, Banking, Lending or customer engagement (preferred)
- GCP Professional Data Engineer or equivalent cloud certification (preferred)
Benefits
Comp & perks- Competitive base salaries
- Bonus incentives
- Support for financial well-being and retirement
- Comprehensive medical, dental, vision, life insurance, and disability benefits (depending on location)
- Flexible working model with hybrid, onsite or virtual arrangements depending on role and business need
- Generous paid parental leave policies (depending on your location)
- Free access to global on-site wellness centers staffed with nurses and doctors (depending on location)
- Free and confidential counseling support through the Healthy Minds program
- Career development and training opportunities