FREE ACCESS
5,000–10,000 jobs/day
See all jobs on Scoutfield
Search thousands of fresh jobs every day.
Discover
- Fresh listings
- Fast filters
- No subscription required
Create a free account and start exploring right away.
Core Competencies
Role fitCore Competencies
Use this summary to align your resume positioning with the role.
Demonstrates expertise in designing and implementing complex data pipelines and data products using SQL, Python, and modern data platforms like Snowflake and Databricks. Proficient in data modeling, quality control, and applying engineering best practices within regulated environments.
Highest-signal resume keywords
SQL DevelopmentPython ProgrammingSnowflake ExperienceDatabricks ProficiencyData Pipeline Design
ATS Keywords
Tailor your resumeApplicant Tracking System Keywords
Tip: use these terms in your resume and cover letter to boost ATS matches.
Hard Skills
SQLPythonData ModelingApache SparkETL/ELT PracticesData Quality ControlBatch ProcessingStreaming Data ProcessingGitAutomated Testing
Soft Skills
Problem SolvingCommunicationCollaborationMentoring
Tools & Technologies
SnowflakeDatabricksApache KafkaDbtApache AirflowAzure Data FactoryTerraformLinux/Unix
Industry Keywords
Data EngineeringRegulated EnvironmentFinancial ServicesData GovernanceMetadata Management
Tech Stack
Tools & technologiesAirflowApacheAzureCloudCyber SecurityETLKafkaLinuxPySparkPythonSnowflakeSparkSQLTerraformUnix
About the role
Key responsibilities & impact- Design, develop, test, deploy and support complex production data pipelines and data products
- Develop maintainable data-processing and transformation solutions using SQL and programming languages such as Python
- Implement solutions using Snowflake, Databricks, dbt and comparable modern data platforms and frameworks
- Build data solutions across relational databases, cloud object storage, warehouse and lakehouse architectures, including open table formats such as Apache Iceberg where applicable
- Develop batch, micro-batch, CDC and streaming pipelines using approved enterprise patterns and technologies such as Apache Kafka or comparable platforms
- Integrate data from relational and mainframe systems, APIs, event streams, SaaS platforms, files and object storage
- Apply normalized, dimensional and denormalized data-modeling techniques for analytical and data-product requirements
- Tune SQL, Snowflake and Databricks/Spark workloads for performance and cost
- Implement data quality, reconciliation, metadata, lineage, logging, monitoring and alerting controls
- Build pipelines supporting error handling, restartability, idempotency, retry, replay and recovery
- Participate in production support, incident resolution and root-cause analysis; implement corrective actions
- Use Git, peer review, automated testing, CI/CD and environment promotion practices
- Develop and reuse shared components, templates and patterns; contribute to engineering standards and documentation
- Participate in technical design and code reviews; identify implementation risks and escalate architectural concerns
- Partner with product owners, architects, governance, cybersecurity, platform and consuming teams
- Contribute to technical epics and stories, estimate assigned work, identify dependencies and deliver through Agile practices
- Provide technical assistance and informal mentoring to less-experienced engineers without direct people-management responsibility
- Adhere to risk and regulatory standards, policies and controls
- Design, implement, maintain and enhance internal controls; identify risk issues requiring escalation
- Maintain internal control standards and implement audit and regulatory issues as applicable
- Promote an environment supporting belonging and reflecting the Company brand
- Complete other related duties as assigned
Requirements
What you’ll need- Associate’s degree and a minimum of 5 years’ systems analysis and/or application development work experience, or Bachelor’s degree and a minimum of 3 years’ systems analysis and/or application development work experience
- In lieu of degree, a combined minimum of 7 years’ education and/or relevant work experience, including a minimum of 3 years’ systems analysis and/or application development work experience
- Strong SQL skills, including development of complex transformations and analytical queries
- Experience with Python or another general-purpose programming language used for data engineering
- Hands-on experience with Snowflake, Databricks or a comparable cloud data platform
- Experience with ELT/ETL and transformation practices, including dbt or a comparable framework
- Experience with relational data stores and warehouse or lakehouse data-modeling concepts
- Experience with distributed processing using Apache Spark/PySpark or a comparable framework
- Experience with Git-based source control, automated testing and CI/CD fundamentals
- Experience with batch data processing and working knowledge of streaming or event-driven concepts
- Knowledge of cloud and object-storage concepts, preferably in Microsoft Azure
- Experience working with structured and semi-structured data using formats such as Parquet, JSON, CSV, Avro and XML
- Experience with Linux/Unix environments and scripting
- Demonstrated ability to independently deliver complex data pipeline components and data products within an established architecture
- Demonstrated ability to troubleshoot data-processing, data-quality and production issues across multiple technologies
- Demonstrated ability to apply established engineering patterns while identifying opportunities for improvement
- Demonstrated ability to communicate implementation choices, risks and dependencies to technical and product partners
- Demonstrated ability to manage assigned work from design through production support with limited day-to-day supervision
- Preferred: Production experience with Snowflake, Databricks, Apache Spark/PySpark and dbt
- Preferred: Experience with Apache Iceberg or another open table format
- Preferred: Experience with Apache Kafka, CDC or event-driven data processing
- Preferred: Experience with orchestration technologies such as Apache Airflow, Azure Data Factory, Databricks Workflows or dbt orchestration
- Preferred: Experience with Terraform or another Infrastructure as Code technology
- Preferred: Experience with data catalogs, metadata, lineage, observability or automated data-quality tooling
- Preferred: Experience delivering governed enterprise data products
- Preferred: Experience working in a regulated environment such as financial services