Senior Data Engineer, Data Lake

Visa

. Develop, test, and maintain data ingestion, transformation, and quality-check pipelines using PySpark/SparkSQL on Databricks .

Posted 9/30/2026full-timeRemote • BrazilSeniorWebsite

Core Competencies

Role fit

Core Competencies

Use this summary to align your resume positioning with the role.

Demonstrates expertise in developing and maintaining data ingestion and transformation pipelines using PySpark and SQL, with a strong focus on data quality and orchestration using Airflow. Proficient in cloud data lake design and familiar with CI/CD practices and documentation standards.

Highest-signal resume keywords

PySpark/SparkSQLData Pipeline ConceptsAirflow DAGsSQL OptimizationDatabricks

ATS Keywords

Tailor your resume

Applicant Tracking System Keywords

Tip: use these terms in your resume and cover letter to boost ATS matches.

Hard Skills

PythonSQLData ModelingData Quality ChecksETL/ELT PatternsAmazon S3Git/GitHubDelta LakeApache AirflowTerraform

Soft Skills

CollaborationProblem-SolvingCommunication

Tools & Technologies

DatabricksGreat ExpectationsMWAACloudWatchBI Integration

Industry Keywords

Data LakeDimensional ModelingKimballCI/CDData Governance

Tech Stack

Tools & technologies

AirflowApacheAWSCloudETLPySparkPythonSparkSQLTerraform

About the role

Key responsibilities & impact

Develop, test, and maintain data ingestion, transformation, and quality-check pipelines using PySpark/SparkSQL on Databricks
Build and modify Airflow DAGs (MWAA) for pipeline orchestration
Write and optimize SQL queries for data transformation and validation
Implement and monitor data quality checks using Great Expectations or equivalent
Participate in code reviews
Investigate pipeline failures and data quality issues with guidance from senior engineers
Write and maintain documentation for datasets and pipelines
Participate in sprint planning, estimation, and retrospectives
Follow team-defined CI/CD, testing, and governance standards
Contribute to the reliability and quality of Visa's corporate data lake

Requirements

What you’ll need

Must be based in Brazil
Proficiency in English at B2 level or above (Upper-Intermediate)
3+ years of relevant work experience with a Bachelor's or Associate’s Degree OR 5+ years of relevant work experience
Python for automation and data processing
Data pipeline concepts (batch, ETL/ELT patterns)
Apache Spark basics (PySpark or SparkSQL)
Databricks (jobs, workflows, cluster management, tuning)
SQL (intermediate–advanced: joins, window functions, CTEs)
Amazon S3 data lake design (partitioning, layout, lifecycle)
Basic cloud concepts (AWS: S3, IAM, CloudWatch)
Data modeling (dimensional / Kimball, medallion layers)
Git/GitHub workflow and code review
Preferred: Delta Lake, Apache Airflow/MWAA, CDC patterns, data quality concepts, Terraform basics, BI integration (Superset, dashboarding))

Benefits

Comp & perks

Remote work arrangement
Opportunity to create impact at scale
Skill growth and professional development through meaningful work
Scheduled-notice Visa office presence may be available for remote positions