FREE ACCESS
5,000–10,000 jobs/day
See all jobs on Scoutfield
Search thousands of fresh jobs every day.
Discover
- Fresh listings
- Fast filters
- No subscription required
Create a free account and start exploring right away.
Core Competencies
Role fitCore Competencies
Use this summary to align your resume positioning with the role.
Demonstrates expertise in designing and developing large-scale data pipelines using Python and Spark, with a strong understanding of cloud and on-premise data solutions. Proficient in data ingestion, optimization techniques, and collaboration with data scientists to drive AI projects.
Highest-signal resume keywords
Data EngineeringPython DevelopmentSpark for Data ProcessingData Pipeline DevelopmentAWS Data Services
ATS Keywords
Tailor your resumeApplicant Tracking System Keywords
Tip: use these terms in your resume and cover letter to boost ATS matches.
Hard Skills
Data EngineeringPythonObject-Oriented ProgrammingSparkDbtAirflowKubeflowDockerAWSMachine Learning
Soft Skills
Team PlayerSelf-Learning AbilityAnalytical SkillsInitiativeAbility to Work Under Pressure
Tools & Technologies
AirflowKubeflowDockerAWS AthenaAWS GlueAWS EMRAWS RedshiftAWS RDSQuickSightTableau
Industry Keywords
Data WarehouseData PipelinesCloud SolutionsOn-Premise SolutionsData FormatsParquetIcebergAvroHDF5Delta Lake
Tech Stack
Tools & technologiesAirflowAmazon RedshiftAWSCloudDockerKubernetesPythonSparkTableau
About the role
Key responsibilities & impact- Integrate into the development team for a core DWH project and various AI projects
- Ingest large volumes of new data
- Conduct deep understanding and examination of data in close collaboration with Data Scientists
- Design and develop critical, diverse, and large-scale data pipelines and processes
- Deploy and optimize data solutions across cloud and on-premise environments
- Work with development, production, and end-user interfaces
Requirements
What you’ll need- B.Sc. in Computer Science, Engineering, Mathematics, or Statistics - Mandatory
- 3+ years of experience as a Data Engineer
- 3+ years of development experience in Python - Mandatory
- Hands-on experience with Object-Oriented Programming (OOP)
- Hands-on experience with Spark for large-scale data processing - Mandatory
- Hands-on experience with dbt
- Experience building and maintaining data pipelines using tools like Airflow or Kubeflow
- Practical experience and understanding of Docker and container platforms such as ECS / Kubernetes
- Understanding of optimization techniques and advanced data formats, including Parquet, Iceberg, Avro, HDF5, and Delta Lake
- Hands-on experience with AWS data services, including Athena, Glue, EMR / EMR Serverless, Redshift, and RDS - Significant advantage
- Experience integrating BI tools such as QuickSight, Tableau, and PowerBI
- Understanding of Machine Learning concepts and processes
- Practical knowledge of GenAI / prompt engineering solutions - Advantage
- Initiative and independence; team player
- Self-learning ability and high analytical skills
- Ability to work under pressure, with a high level of responsibility and commitment
Benefits
Comp & perks- Competitive salaries
- Bonus incentives
- Benefits
- Flexible hours
- Mentoring
- Learning and personal development opportunities