FREE ACCESS
5,000–10,000 jobs/day
See all jobs on Scoutfield
Search thousands of fresh jobs every day.
Discover
- Fresh listings
- Fast filters
- No subscription required
Create a free account and start exploring right away.
Core Competencies
Role fitCore Competencies
Use this summary to align your resume positioning with the role.
Demonstrates expertise in developing and maintaining data ingestion pipelines using Databricks Autoloader, implementing large-scale data processing solutions with Spark Structured Streaming and Spark SQL, and applying data quality practices. Proficient in optimizing performance and costs in Databricks environments while collaborating effectively with business and analytics teams.
Highest-signal resume keywords
Data EngineeringPython ProgrammingApache SparkDatabricksData Quality Practices
ATS Keywords
Tailor your resumeApplicant Tracking System Keywords
Tip: use these terms in your resume and cover letter to boost ATS matches.
Hard Skills
Data Ingestion PipelinesSpark Structured StreamingSpark SQLDelta LakeProcessing OptimizationIncremental ProcessingSchema EvolutionTechnical DocumentationAgile MethodologiesETL/ELT Processes
Soft Skills
Analytical SkillsProblem-SolvingCommunication SkillsSelf-DirectedInfluencing Decisions
Tools & Technologies
GitDatabricks AutoloaderVersion Control
Industry Keywords
Large-Scale Data ProcessingData AvailabilityData StructureData SecurityScalable Data Solutions
Tech Stack
Tools & technologiesApacheETLPythonSparkSQL
About the role
Key responsibilities & impact- Develop and maintain data ingestion pipelines using Databricks Autoloader
- Implement incremental processing and schema evolution
- Implement large-scale data processing solutions using Spark Structured Streaming and Spark SQL
- Apply partitioning, processing optimization, and Delta Lake
- Analyze and optimize the performance and costs of Databricks environments
- Implement data quality practices
- Work with Git and pull request processes
- Prepare technical documentation for developed solutions
- Collaborate with business and Analytics teams to ensure data availability, structure, and security
- Develop and enhance scalable, efficient, secure, and high-performance data solutions
Requirements
What you’ll need- Experience in data engineering and processing large volumes of information
- Proficiency in Python and Apache Spark
- Experience with Databricks
- Knowledge of building and maintaining data ingestion pipelines
- Experience with Spark Structured Streaming and Spark SQL
- Knowledge of Delta Lake and processing optimization techniques
- Experience with Git and version control
- Knowledge of data quality practices
- Preferred: knowledge of modern data architectures and ETL/ELT processes
- Preferred: experience with agile methodologies such as Scrum and Kanban
- Preferred: experience optimizing costs and performance in scalable data environments
- Analytical, self-directed, and skilled at problem-solving
- Eagerness to learn new technologies and propose innovative solutions
- Excellent communication skills with technical and non-technical audiences
- Ability to translate business needs into practical, scalable solutions
- Ability to influence decisions and build trusted relationships with stakeholders
Benefits
Comp & perks- Remote work environment
- Opportunities to develop skills and technical judgment
- Collaboration with international teams across multiple geographies
- Diversity and multiculturalism culture
- Knowledge sharing and continuous improvement