FREE ACCESS
5,000–10,000 jobs/day
See all jobs on Scoutfield
Search thousands of fresh jobs every day.
Discover
- Fresh listings
- Fast filters
- No subscription required
Create a free account and start exploring right away.
Core Competencies
Role fitCore Competencies
Use this summary to align your resume positioning with the role.
Demonstrates extensive experience in data engineering with a strong focus on implementing scalable data pipelines using Azure Data Factory and Apache Airflow. Proficient in developing data transformations with PySpark and SQL, while ensuring operational readiness and optimizing performance across data workloads.
Highest-signal resume keywords
Data Engineering ExperiencePython (PySpark) ExpertiseAdvanced SQL SkillsAzure Data Factory ProficiencyApache Airflow Experience
ATS Keywords
Tailor your resumeApplicant Tracking System Keywords
Tip: use these terms in your resume and cover letter to boost ATS matches.
Hard Skills
Data Pipeline ImplementationData Transformation DevelopmentPerformance OptimizationData Modelling FundamentalsETL/ELT Design PatternsBatch and Streaming PipelinesMedallion Architecture KnowledgeMetadata-Driven ArchitectureData Warehouse ArchitectureData Lakehouse Architecture
Soft Skills
CollaborationProblem-SolvingTechnical Decision-MakingRisk DetectionDocumentation
Tools & Technologies
Azure DatabricksUnity CatalogPower BITableauAzure Data Platform
Industry Keywords
Data ProductsIncident ManagementRoot Cause AnalysisOperational ProcessesRunbooks
Tech Stack
Tools & technologiesAirflowApacheAzureETLPySparkPythonSQLTableauUnity
About the role
Key responsibilities & impact- Implement scalable, reliable and reusable data pipelines using Azure Data Factory and Apache Airflow
- Develop data transformations in Azure Databricks using PySpark and SQL
- Create and extend data products following Bronze, Silver, and Gold Medallion Architecture layers
- Optimize performance, cost, and reliability of data workloads
- Raise important questions during business maturation to detect risks before development
- Participate in architecture reviews and technical decision-making
- Promote engineering excellence through code reviews and design reviews
- Collaborate with Architects and Product Owners in data product technical maturation
- Support the Run/Operations team for production pipelines and data products
- Troubleshoot and resolve data incidents, failures, and performance issues
- Participate in incident management, root cause analysis, and post-mortems
- Ensure proper handover, documentation, and operational readiness of developed solutions
- Contribute to improving runbooks, monitoring, and operational processes
Requirements
What you’ll need- 6+ years of experience in data engineering
- Strong technical expertise in Python (PySpark)
- Advanced SQL skills
- Hands-on experience with Azure-based Data Platform
- Experience with Databricks
- Experience with Unity Catalog
- Experience with Apache Airflow
- Experience with Azure Data Factory
- Experience with reporting and BI tools such as Power BI, Tableau or similar
- Solid understanding of data modelling fundamentals, main architectures and design patterns
- Knowledge of Data Warehouse, Data Lake and Data Lakehouse architectures
- Knowledge of Medallion architecture
- Knowledge of metadata-driven architecture
- Knowledge of ETL/ELT design patterns
- Understanding of partitioning, optimization, and performance
- Understanding of batch and streaming pipelines
