ShyftLabs

Data Engineer

• Design, develop, and maintain scalable ETL/ELT pipelines using Databricks, PySpark, and SQL
• Integrate data from multiple sources, including databases, Amazon S3, files, and REST APIs
• Build data pipelines with Databricks Unity Catalog
• Implement business logic, data transformations, and dimensional data models
• Create, schedule, monitor, and optimize Databricks Jobs and Workflows
• Design and manage Delta Lake tables using Medallion Architecture (Bronze, Silver, Gold)
• Ensure data quality through validations, error handling, logging, and monitoring
• Optimize Spark workloads for performance, scalability, and reliability
• Collaborate with cross-functional teams to deliver production-ready data solutions

Data Engineer – Databricks

• Design, develop, and maintain scalable ETL/ELT pipelines using Databricks, PySpark, and SQL
• Integrate data from multiple sources, including databases, Amazon S3, files, and REST APIs
• Build data pipelines with Databricks Unity Catalog
• Implement business logic, data transformations, and dimensional data models
• Create, schedule, monitor, and optimize Databricks Jobs and Workflows
• Design and manage Delta Lake tables using Medallion Architecture (Bronze, Silver, Gold)
• Ensure data quality through validations, error handling, logging, and monitoring
• Optimize Spark workloads for performance, scalability, and reliability
• Collaborate with cross-functional teams to deliver production-ready data solutions
• Guide engineers and drive technical decisions in a lead capacity

Data Engineer – Databricks

Lead Data Engineer – Databricks