FREE ACCESS
5,000–10,000 jobs/day
See all jobs on Scoutfield
Search thousands of fresh jobs every day.
Discover
- Fresh listings
- Fast filters
- No subscription required
Create a free account and start exploring right away.
Core Competencies
Role fitCore Competencies
Use this summary to align your resume positioning with the role.
Demonstrates expertise in designing and optimizing data platform architectures using AWS and Databricks, with a strong focus on building automated data pipelines and managing Delta Lake lakehouses. Proficient in mentoring data engineering teams and implementing best practices for CI/CD and data governance.
Highest-signal resume keywords
AWS Data LakesDatabricks PlatformDelta Lake InternalsPythonSQL
ATS Keywords
Tailor your resumeApplicant Tracking System Keywords
Tip: use these terms in your resume and cover letter to boost ATS matches.
Hard Skills
Data EngineeringData ArchitectureAWS GlueDatabricks JobsDelta Live TablesMLOpsREST API IntegrationsSQL ServerData IngestionData Transformation
Soft Skills
MentoringCollaborationProblem-Solving
Tools & Technologies
Databricks Unity CatalogDatabricks SQL WarehousesCI/CDVersion ControlAutomated Testing
Industry Keywords
Big Data PlatformsCloud-to-Lakehouse MigrationsData GovernanceMetadata ManagementAccess Control
Tech Stack
Tools & technologiesAWSCloudPySparkPythonSQLUnity
About the role
Key responsibilities & impact- Design, build, and optimize scalable, resilient, and secure data platform architectures using AWS and Databricks
- Build automated pipelines using AWS Glue, Databricks Jobs/Workflows, JDBC ingestion from multi-tenant SQL Server sources, and REST API integrations (HubSpot, Jira, Sympla, Zoom)
- Architect and maintain the Delta Lake lakehouse across bronze, silver, and gold layers, including ACID transactions, time travel, schema evolution, and Z-ORDER/partitioning strategies for cost and performance optimization
- Architect and maintain granular access controls, data lineage, and metadata management through Databricks Unity Catalog across multiple workspaces, including secret scope management and credential rotation
- Build and operationalize data infrastructure for ML/AI and LLM workloads
- Configure Databricks AI/BI Genie spaces, including trusted assets, instructions, and metric spaces, to support self-service BI in natural language
- Monitor and optimize Databricks SQL Warehouses and job clusters for performance and cost, including allocation through system.billing, warehouse sizing, idle resource detection, and query tuning
- Guide and mentor senior and mid-level data engineers, establishing engineering best practices for CI/CD, automated testing, and version control
Requirements
What you’ll need- 7+ years of professional experience in data engineering, data architecture, or big data platforms
- Advanced knowledge of AWS Data Lakes, including experience with cloud-to-lakehouse migrations
- Extensive hands-on experience building complex ingestion and transformation workflows using AWS Glue and Databricks Jobs
- Strong knowledge of Delta Lake internals, including merge/upsert patterns, table optimization (OPTIMIZE, Z-ORDER, VACUUM), and schema management at scale
- Proficiency with the Databricks platform, including PySpark, Delta Live Tables, and Serverless SQL Warehouses
- Hands-on experience configuring and managing Databricks Unity Catalog for centralized metadata management, access control, and auditing across multiple workspaces
- Experience building data platforms for ML workflows, MLOps, and conversational AI tools—specifically Databricks Genie—including curating Genie spaces, refining prompts/instructions, and integrating with Unity Catalog semantics
- Advanced proficiency in Python and SQL
Benefits
Comp & perks- Dental Plan | Amil Dental
- Health Plan | Bradesco Saúde
- Mental Health Platform
- Telemedicine Platform
- Life Insurance | Prudential
- Pharmacy Benefit
- Gym Allowance
- Birthday Day Off
- Flexible Meal Voucher
- Home Office Allowance
- Childcare Assistance
- Transportation Voucher and Company Shuttle
- Discount Platform (Allya, SwileShop, and Dell)
- Education Benefits
- Travel Benefits
- Pet Health Plan
- SESC Benefits
- Profit-Sharing Program (PPRL)