FREE ACCESS
5,000–10,000 jobs/day
See all jobs on Scoutfield
Search thousands of fresh jobs every day.
Discover
- Fresh listings
- Fast filters
- No subscription required
Create a free account and start exploring right away.
Senior Data Engineer
73 Strings. Build and operate pipelines, integrations and the warehouse supporting valuation and monitoring .
Core Competencies
Role fitCore Competencies
Use this summary to align your resume positioning with the role.
Expertise in building and operating data pipelines, including batch and streaming processes, with a strong focus on Snowflake and Databricks. Proficient in Python and SQL for pipeline development, alongside experience in CI/CD practices and data quality management.
Highest-signal resume keywords
Data EngineeringSnowflakePythonCI/CDChange Data Capture
ATS Keywords
Tailor your resumeApplicant Tracking System Keywords
Tip: use these terms in your resume and cover letter to boost ATS matches.
Hard Skills
Data Pipeline DevelopmentSQLChange Data CaptureEvent ProcessingData Quality ManagementPerformance TuningDimensional ModelingIncremental LoadingData ReconciliationMonitoring
Soft Skills
CollaborationClient EngagementProblem Solving
Tools & Technologies
DatabricksGitHubAzureApache AirflowKafkaSpark Structured StreamingDebeziumConfluent KafkaDelta SharingDbt
Industry Keywords
Production SystemsMulti-Tenant Data PlatformsData ProtectionValuationsPrivate Markets Data
Tech Stack
Tools & technologiesAirflowApacheAzureKafkaMS SQL ServerPythonSparkSQL
About the role
Key responsibilities & impact- Build and operate pipelines, integrations and the warehouse supporting valuation and monitoring
- Own production pipelines from source capture through transformation, reconciliation and client delivery
- Put production pipelines under GitHub, automated testing and CI/CD
- Help redefine platform architecture across ingestion, processing and delivery
- Build and operate batch and streaming pipelines from databases, APIs, event streams and semi-structured sources
- Implement change data capture and incremental loading, including ordering, deletes, replay and slowly changing dimensions
- Build medallion datasets and dimensional models for Snowflake, Microsoft SQL Server and Databricks
- Apply data contracts, reconciliation and row-level quarantine before publication
- Own GitHub workflow and CI/CD, including tests, reviews, environment promotion and deployment as code
- Investigate production data failures and translate product, valuation and client-facing requirements into operable pipelines
Requirements
What you’ll need- 10+ years in data engineering on production systems
- Snowflake or Databricks as a primary platform, including modelling, performance tuning and cost management
- Python and SQL for pipeline development and testing
- Change data capture and event processing, including ordering, replay and schema change
- Azure, including Databricks, ADLS and private network connectivity
- GitHub and CI/CD for data workloads, using GitHub Actions or an equivalent system
- Data quality, reconciliation, monitoring and production incident response
- Experience building multi-tenant, secure data platforms, including tenant isolation, access control and data protection
- Desirable: Databricks Lakeflow, Auto CDC, Declarative Automation Bundles and DQX, or Snowflake equivalents
- Desirable: Debezium, Kafka Connect or Confluent Kafka
- Desirable: Apache Airflow or equivalent workflow orchestrator
- Desirable: Kafka or Spark Structured Streaming, Apache Iceberg or Delta Sharing, and dbt
- Desirable: Private markets data including valuations, funds, portfolio companies or capital activity
- Comfortable working directly with client technical teams and collaborating across field engineering, product and other stakeholders
Benefits
Comp & perks- Supportive environment
- Opportunities to take initiative
- Continuous learning opportunities
- Collaborative culture