FREE ACCESS
5,000–10,000 jobs/day
See all jobs on Scoutfield
Search thousands of fresh jobs every day.
Discover
- Fresh listings
- Fast filters
- No subscription required
Create a free account and start exploring right away.
Core Competencies
Role fitCore Competencies
Use this summary to align your resume positioning with the role.
Demonstrates expertise in designing and implementing scalable data solutions, optimizing data pipelines, and ensuring data quality and governance. Proven ability to lead and mentor technical teams while fostering collaboration and continuous improvement.
Highest-signal resume keywords
Data Pipeline DevelopmentTechnical LeadershipApache SparkSQL and NoSQL DatabasesAutomated Testing and CI/CD
ATS Keywords
Tailor your resumeApplicant Tracking System Keywords
Tip: use these terms in your resume and cover letter to boost ATS matches.
Hard Skills
PythonJavaScalaData ModelingData ArchitectureBatch and Streaming Data SolutionsIntegration from APIs and DatabasesObservabilityDevOps PracticesPerformance Optimization
Soft Skills
CommunicationCollaborationDecision-MakingStakeholder ManagementMentorship
Tools & Technologies
Apache KafkaApache FlinkDockerKubernetesDbt
Industry Keywords
Data GovernanceData QualityCloud ModernizationInfrastructure AutomationRegulatory Compliance
Tech Stack
Tools & technologiesApacheCloudDockerJavaKafkaKubernetesNoSQLPythonScalaSparkSQL
About the role
Key responsibilities & impact- Lead, mentor, and coach Data Engineers and support their technical and professional development
- Establish engineering standards and promote best practices, collaboration, accountability, knowledge sharing, and continuous improvement
- Lead the design, development, implementation, and support of scalable batch and streaming data solutions
- Develop reusable patterns for data ingestion, transformation, storage, and access
- Integrate data from APIs, databases, event streams, third-party platforms, and enterprise systems
- Participate in architecture discussions, solution design, code reviews, troubleshooting, and complex problem-solving
- Promote version control, automated testing, continuous integration, automated deployment, and infrastructure automation
- Establish standards for data quality, validation, monitoring, lineage, and observability
- Optimize data pipelines, databases, queries, and storage solutions for performance and cost
- Support production operations and lead critical incidents, root-cause analysis, and preventative improvements
- Partner with Product Managers, Architects, Engineers, Analysts, and business stakeholders to deliver data solutions
- Translate business and product needs into technical roadmaps and defined engineering work
- Lead or contribute to planning, prioritization, estimation, and delivery activities
- Manage risks, dependencies, resource needs, and competing priorities
- Communicate progress, technical recommendations, risks, and dependencies to stakeholders and leaders
- Promote data governance, metadata management, data lineage, classification, and data quality
- Ensure compliance with security, privacy, governance, and regulatory requirements
- Support cloud modernization, data platform transformation, innovation, automation, and responsible use of generative and agent-based AI tools
- Depending on position level, support or lead hiring, onboarding, performance feedback, career development, and workforce planning
- This job has supervisory duties
Requirements
What you’ll need- 5 or more years of experience designing, developing, and supporting data solutions, data pipelines, or enterprise software systems (5+ years opportunity)
- 1 or more years of experience providing technical leadership, mentorship, coaching, or project leadership (5+ years opportunity)
- 7 or more years of experience designing, developing, and supporting data solutions, data pipelines, or enterprise software systems (7+ years opportunity)
- 3 or more years of experience leading, mentoring, or managing technical professionals (7+ years opportunity)
- Experience guiding engineers through technical mentorship and knowledge sharing
- Experience with coaching, performance management, career development, hiring, and team development
- Strong hands-on experience developing data pipelines using Python, Java, Scala, or a comparable programming language
- Experience designing and supporting batch or streaming data solutions
- Experience with Apache Spark, Apache Kafka, Apache Flink, dbt, or comparable technologies
- Strong knowledge of SQL, NoSQL databases, data modeling, and data storage or architecture concepts
- Experience integrating data from APIs, databases, event streams, third-party platforms, and enterprise systems
- Experience with automated testing, continuous integration and delivery, deployment automation, observability, and DevOps practices
- Experience with Docker and Kubernetes
- Strong communication, collaboration, decision-making, and stakeholder management skills
- Ability to influence technical outcomes and balance hands-on engineering, people leadership, and delivery accountability
- Background investigation required
- Reliable internet with minimum speeds of 50 MB download and 5 MB upload
- Dedicated, private home workspace with appropriate desk and seating
- Allstate generally does not sponsor individuals for employment-based visas for this position
Benefits
Comp & perks- Comprehensive technology setup, including a laptop, monitors, headset, keyboard, and mouse
- Monthly connectivity reimbursement for employees eligible to work from home
- Remote work arrangement