FREE ACCESS
5,000–10,000 jobs/day
See all jobs on Scoutfield
Search thousands of fresh jobs every day.
Discover
- Fresh listings
- Fast filters
- No subscription required
Create a free account and start exploring right away.
Tech Stack
Tools & technologiesJavaPostgresPySparkPythonSparkSpringSpring BootSpringBootSQLTerraform
About the role
Key responsibilities & impact- Own the data platform end-to-end, including ingestion, transformation, storage, query, and access
- Drive the data platform roadmap as company data needs grow
- Build and evolve batch and streaming pipelines using PySpark/EMR, Kinesis, Lambda, and Step Functions
- Ingest data from Postgres, Salesforce, third-party vendors, and product event streams into an Iceberg-based lake
- Design SCD tables, event tables, and data conventions for engineers and analysts
- Partner with product, engineering, analytics, and operations stakeholders to turn data requests into reliable pipelines
- Write documentation and tooling that enable self-service
- Own data-system security and compliance, including audit logging, access control, and temporary-access workflows
- Optimize backend query performance through read-replica routing, indexing, caching, and I/O instrumentation in Java/Spring services
- Investigate and remediate IOPS spikes, pipeline failures, schema drift, and late data
- Use AI for pipeline scaffolding, schema work, and investigations; ship internal AI tooling
- Mentor engineers and analysts on working with the data platform
Requirements
What you’ll need- 5+ years of experience building production data pipelines and platforms
- Deep Python (PySpark) and SQL fluency, including tuning Spark jobs at scale
- Skills and willingness to work on services around the data layer (Java, Spring Boot)
- Hands-on experience with distributed compute (Spark/EMR), streaming (Kinesis), and object storage (S3)
- Solid Postgres fundamentals: query optimization, indexing, replication, replica routing, and identifying database bottlenecks
- Experience with Iceberg and Trino, or similar
- Comfort with CI/CD and Terraform
- Experience building with AI, including frontier models or agentic coding tools
- Track record of working across product, operations, and other teams
- Experience handling security and PII/PHI compliance as core engineering work
Benefits
Comp & perks- Professional growth opportunities
- Best Places to Work recognition (company recognition, not an individual compensation benefit)