FREE ACCESS
5,000–10,000 jobs/day
See all jobs on Scoutfield
Search thousands of fresh jobs every day.
Discover
- Fresh listings
- Fast filters
- No subscription required
Create a free account and start exploring right away.
Data Engineer, Data Platforms
S&P Global. Partner with Databricks and platform teams to transition existing pipelines into the target enterprise data platform .
Core Competencies
Role fitCore Competencies
Use this summary to align your resume positioning with the role.
Demonstrates expertise in data engineering and pipeline optimization, with a strong focus on AWS technologies and Databricks platforms. Capable of implementing robust data quality controls and observability practices while collaborating effectively with global teams.
Highest-signal resume keywords
Data EngineeringAWS TechnologiesDatabricks PlatformsPipeline OptimizationData Quality Control
ATS Keywords
Tailor your resumeApplicant Tracking System Keywords
Tip: use these terms in your resume and cover letter to boost ATS matches.
Hard Skills
Batch Data PipelinesStreaming Data PipelinesDelta LakeApache IcebergAWS GlueAWS LambdaAmazon KinesisData Quality ManagementMetadata ManagementSchema Management
Soft Skills
Strong Communication SkillsCollaboration Across Time ZonesTechnical Guidance
Tools & Technologies
Databricks Unity CatalogAWS IAMClaudeGitHub CoPilotLLM-based Development Features
Certifications & Qualifications
Databricks CertificationAWS Certification
Industry Keywords
Enterprise Data PlatformsLakehouse ArchitectureData GovernanceGolden Record ConceptsRegulated Environments
Tech Stack
Tools & technologiesApacheAWSUnity
About the role
Key responsibilities & impact- Partner with Databricks and platform teams to transition existing pipelines into the target enterprise data platform
- Provide hands-on engineering support to stabilize and optimize pipelines for performance, scalability, maintainability, and operational reliability
- Implement repeatable engineering patterns for ingestion, transformation, testing, deployment, and monitoring
- Support secure AWS deployment patterns and governed data access using AWS IAM, encryption, and platform guardrails
- Work with Delta Lake, Apache Iceberg, Databricks Unity Catalog, and metadata-driven pipeline design
- Establish and maintain observability practices including logging, telemetry, alerts, and dashboards
- Support asset-agnostic onboarding of new datasets using standard ingestion and transformation frameworks
- Help offshore teams adopt common onboarding standards, delivery practices, and technical templates
- Create reusable reference implementations, onboarding templates, documentation, and operational runbooks
- Support batch and streaming onboarding patterns using AWS Glue, AWS Lambda, Amazon Kinesis, or event-driven integrations
- Support integration with the enterprise data mastering platform
- Maintain data quality controls, reconciliation, and metadata alignment
- Support golden record/golden copy generation and maintenance for key financial datasets
- Provide technical guidance through code reviews, design reviews, and implementation planning
- Contribute to standards across CI/CD, infrastructure automation, testing, and operational readiness
- Act as a technical bridge between offshore execution teams and global platform stakeholders
- Use Claude, GitHub CoPilot, and LLMs for code drafting, refactoring, documentation, unit tests, and troubleshooting
- Apply context engineering and validate AI-generated outputs for correctness, security, and maintainability
- Follow responsible AI practices protecting sensitive data, credentials, and regulated information
Requirements
What you’ll need- 5–7+ years of experience in data engineering, data platforms, or related technical domains
- Hands-on experience building and operating batch and/or streaming data pipelines
- Experience working with Databricks-based platforms and lakehouse architectures
- Strong familiarity with Amazon S3, AWS Glue, AWS Lambda, AWS Lake Formation, Amazon Kinesis, and AWS IAM
- Working knowledge of Delta Lake, Apache Iceberg, and Databricks Unity Catalog or equivalent governed lakehouse concepts
- Understanding of data quality, schema management, lineage/metadata, access control, encryption, and operational support
- Ability to provide technical guidance without people management responsibility
- Experience collaborating with global teams across time zones and resolving ambiguous delivery problems
- Strong communication skills and ability to translate technical requirements into implementable tasks
- Practical experience using Claude, GitHub CoPilot, or LLM-based development features
- Ability to use AI with basic context engineering practices
- Ability to validate AI-generated outputs and follow responsible, secure usage practices
- Experience with enterprise onboarding frameworks, reusable ingestion patterns, and pipeline templates is preferred
- Familiarity with data cataloging/metadata management and governed data product concepts is preferred
- Knowledge of semantic modeling is a plus
- Exposure to enterprise data mastering/MDM workflows and golden record concepts is preferred
- Experience in regulated environments is preferred
- Databricks and/or AWS certifications are a plus
Benefits
Comp & perks- Health care coverage designed for the mind and body
- Generous time off
- Continuous learning resources and career development opportunities
- Competitive pay
- Retirement planning
- Continuing education program
- Company-matched student loan contribution
- Financial wellness programs
- Family-friendly benefits and perks
- Retail discounts
- Referral incentive awards