FREE ACCESS
5,000–10,000 jobs/day
See all jobs on Scoutfield
Search thousands of fresh jobs every day.
Discover
- Fresh listings
- Fast filters
- No subscription required
Create a free account and start exploring right away.
Data Engineer, Data Platforms
S&P Global. Partner with Databricks and platform teams to transition existing pipelines into the target enterprise data platform .
Core Competencies
Role fitCore Competencies
Use this summary to align your resume positioning with the role.
Demonstrates expertise in data engineering and pipeline optimization, with a strong focus on AWS technologies and Databricks platforms. Capable of implementing robust data quality controls and governance practices while collaborating effectively across global teams.
Highest-signal resume keywords
Data EngineeringAWS GlueDatabricks Unity CatalogBatch And Streaming Data PipelinesData Quality Management
ATS Keywords
Tailor your resumeApplicant Tracking System Keywords
Tip: use these terms in your resume and cover letter to boost ATS matches.
Hard Skills
Data Pipeline DevelopmentMetadata ManagementSchema ManagementOperational SupportAI Tools UsageContext EngineeringGolden Record GenerationIngestion PatternsTelemetry PracticesCI/CD Standards
Soft Skills
Strong CommunicationTechnical GuidanceCollaboration Across Time ZonesProblem-SolvingDocumentation Quality
Tools & Technologies
AWS IAMAmazon S3AWS LambdaAWS Lake FormationAmazon KinesisDelta LakeApache IcebergDatabricksAI Development ToolsEvent-Driven Integrations
Certifications & Qualifications
Databricks CertificationAWS Certification
Industry Keywords
Data PlatformsLakehouse ArchitectureGoverned Data AccessEnterprise Data MasteringRegulated Environments
Tech Stack
Tools & technologiesApacheAWSUnity
About the role
Key responsibilities & impact- Partner with Databricks and platform teams to transition existing pipelines into the target enterprise data platform
- Provide hands-on engineering support to stabilize and optimize pipelines post-transition
- Implement repeatable engineering patterns for ingestion, transformation, testing, deployment, and monitoring
- Support secure AWS deployment patterns and governed data access using AWS IAM, encryption, and platform guardrails
- Work with Delta Lake, Apache Iceberg, Databricks Unity Catalog, and metadata-driven pipeline design
- Establish and maintain observability practices including logging, telemetry, alerts, and dashboards
- Support asset-agnostic onboarding using standard ingestion and transformation frameworks
- Help offshore teams adopt common onboarding standards, delivery practices, and technical templates
- Contribute reusable technical assets including reference implementations, onboarding templates, documentation, and operational runbooks
- Support batch and streaming onboarding patterns using AWS Glue, AWS Lambda, Amazon Kinesis, or event-driven integrations
- Support integration of platform datasets with the enterprise data mastering platform
- Maintain data quality controls, reconciliation, and metadata alignment for trusted mastered data
- Support golden record/golden copy generation and maintenance for key financial datasets
- Provide technical guidance through code reviews, design reviews, and implementation planning
- Contribute to standards across CI/CD, infrastructure automation, testing practices, and operational readiness
- Ensure documentation quality and operational discipline
- Act as a technical bridge between offshore execution teams and global platform stakeholders
- Use AI tools to draft code, refactor, write documentation, generate unit tests, and troubleshoot
- Apply prompt engineering and context engineering practices
- Validate AI-generated outputs for correctness, security, and maintainability
- Follow responsible AI usage expectations
Requirements
What you’ll need- 5–7+ years of experience in data engineering, data platforms, or related technical domains
- Hands-on experience building and operating batch and/or streaming data pipelines
- Experience working with Databricks-based platforms and lakehouse architectures
- Strong familiarity with Amazon S3, AWS Glue, AWS Lambda, AWS Lake Formation, Amazon Kinesis, and AWS IAM
- Working knowledge of Delta Lake, Apache Iceberg, and Databricks Unity Catalog or equivalent governed lakehouse concepts
- Understanding of data quality, schema management, lineage/metadata basics, access control, encryption, and operational support
- Ability to provide technical guidance without people management responsibility
- Experience collaborating with global teams across time zones and working through ambiguous delivery problems
- Strong communication skills and ability to translate technical requirements into implementable tasks
- Practical experience using Claude, GitHub CoPilot, or LLM-based development features
- Ability to use AI effectively with basic context engineering practices
- Ability to validate AI-generated outputs and follow responsible, secure usage practices
- Experience with enterprise onboarding frameworks, reusable ingestion patterns, and pipeline templates
- Familiarity with data cataloging/metadata management and governed data product concepts
- Knowledge of semantic modeling is a plus
- Exposure to enterprise data mastering/MDM workflows, golden record concepts, and trusted data distribution
- Experience in regulated environments with strong governance, auditability, and security expectations
- Databricks and/or AWS certifications or equivalent practical experience are a plus
Benefits
Comp & perks- Health care coverage designed for the mind and body
- Generous time off
- Continuous learning and career development resources
- Competitive pay
- Retirement planning
- Continuing education program
- Company-matched student loan contribution
- Financial wellness programs
- Family benefits and perks
- Retail discounts
- Referral incentive awards