Demonstrates expertise in designing and optimizing ETL/ELT pipelines using Apache Spark and Delta Lake across cloud platforms, with a strong focus on automation, data security, and compliance. Proficient in collaborating with cross-functional teams to translate AI data requirements into actionable business outcomes.

Highest-signal resume keywords

Apache SparkDelta LakeDatabricksTerraformCI/CD

Tech Stack

Tools & technologies

ApacheAWSAzureCloudETLGoogle Cloud PlatformPythonSparkSQLTerraformUnity

About the role

Key responsibilities & impact

Design, build, and optimize scalable ETL/ELT pipelines using Apache Spark and Delta Lake across AWS, Azure, and GCP.
Implement scalable Lakehouse architectures for structured and unstructured enterprise data.
Automate data ingestion, storage, and feature-engineering workflows for analytics and real-time AI workloads.
Tune Spark jobs for low latency, high throughput, and cloud cost efficiency.
Partner with ML engineers and data scientists to operationalize AI models in Databricks using MLflow.
Implement CI/CD pipelines and Infrastructure-as-Code using Terraform and Databricks CLI.
Enforce enterprise data security, access controls, and compliance standards including GDPR, HIPAA, and SOC 2 within Unity Catalog and cloud ecosystems.
Collaborate with client technology leaders, product managers, and engineering teams to translate AI data requirements into business outcomes.
Author technical documentation and lead solution handoffs to client engineering teams.
Report to the Product Manager/Engineering Lead.

Requirements

What you’ll need

3+ years in cloud-native data engineering.
2+ years of dedicated, hands-on Databricks, Apache Spark, Delta Lake, and MLflow experience.
High proficiency in Python, SQL, Spark frameworks, CI/CD, and Infrastructure-as-Code (Terraform or Databricks CLI).
Active Databricks Certified Data Engineer (Associate or Professional) or Databricks ML Professional certification is strongly preferred.
Proven track record operating in fast-paced Agile/DevOps consulting environments.
Experience delivering cloud-agnostic architectures.
Exceptional client communication.
Must satisfy U.S. work-authorization requirements for direct W-2 employment through E-Verify.

Benefits

Comp & perks

Paid Time Off – Take the time you need to recharge and stay productive.
Remote-First Working Environment – Collaborate from anywhere while staying connected with our global team.
Comprehensive Health Coverage – Medical, Dental, Vision
401(k) Retirement Plan – Plan for your future with access to a company-sponsored 401(k) program.
Annual Learning & Development Stipend – Invest in your skills with conferences, certifications, or courses.
Peer Mentorship & Coaching – Learn from experienced engineers, product managers, and architects to accelerate your growth.
Professional Growth Opportunities – Exposure to cutting-edge AWS GenAI, data, and cloud technologies across diverse industries.
Company Outings & Volunteer Opportunities – Build relationships and give back to the community.
Collaborative, Innovative Culture – Work alongside top talent in a fast-paced, supportive environment that values curiosity and initiative.

ATS Keywords

✓ Tailor your resume

Applicant Tracking System Keywords

Tip: use these terms in your resume and cover letter to boost ATS matches.

Hard Skills

ETL PipelinesELT PipelinesPythonSQLInfrastructure-as-CodeData IngestionFeature EngineeringCloud Cost EfficiencyAgileDevOps

Soft Skills

Client CommunicationCollaboration

Tools & Technologies

AWSAzureGCPMLflowUnity Catalog

Certifications & Qualifications

Databricks Certified Data EngineerDatabricks ML Professional

Industry Keywords

GDPRHIPAASOC 2Cloud-native Data EngineeringCloud-agnostic Architectures