FREE ACCESS
5,000–10,000 jobs/day
See all jobs on Scoutfield
Search thousands of fresh jobs every day.
Discover
- Fresh listings
- Fast filters
- No subscription required
Create a free account and start exploring right away.
Core Competencies
Role fitCore Competencies
Use this summary to align your resume positioning with the role.
Demonstrates expertise in designing and optimizing scalable data platform architectures using AWS and Databricks, with a strong focus on data ingestion, transformation workflows, and ML/AI workloads. Proficient in managing Delta Lake internals and Databricks Unity Catalog for effective metadata management and access control.
Highest-signal resume keywords
AWS Data LakesDatabricks PlatformDelta Lake InternalsPythonSQL
ATS Keywords
Tailor your resumeApplicant Tracking System Keywords
Tip: use these terms in your resume and cover letter to boost ATS matches.
Hard Skills
Data EngineeringData ArchitectureAWS GlueDatabricks JobsDelta Live TablesMLOpsREST API IntegrationsSchema ManagementAutomated TestingVersion Control
Soft Skills
MentoringCollaborationProblem-Solving
Tools & Technologies
Databricks Unity CatalogSQL ServerHubSpotJiraSymplaZoom
Industry Keywords
Big Data PlatformsLakehouse ArchitecturesData IngestionData TransformationData Lineage
Tech Stack
Tools & technologiesAWSCloudPySparkPythonSQLUnity
About the role
Key responsibilities & impact- Design, build, and optimize scalable, resilient, and secure data platform architectures using AWS and Databricks
- Build automated pipelines using AWS Glue, Databricks Jobs/Workflows, JDBC ingestion from multi-tenant SQL Server sources, and REST API integrations (HubSpot, Jira, Sympla, Zoom)
- Architect and maintain the Delta Lake lakehouse across bronze, silver, and gold layers, including ACID transactions, time travel, schema evolution, and Z-ORDER/partitioning strategies for cost and performance optimization
- Architect and maintain granular access controls, data lineage, and metadata management through Databricks Unity Catalog across multiple workspaces, including secret scope management and credential rotation
- Build and operationalize data infrastructure for ML/AI and LLM workloads
- Configure Databricks AI/BI Genie spaces, including trusted assets, instructions, and metric spaces, to support self-service BI using natural language
- Monitor and optimize Databricks SQL Warehouses and job clusters for performance and cost, including allocation through system.billing, warehouse sizing, idle-resource detection, and query tuning
- Guide and mentor senior and mid-level data engineers, establishing engineering best practices in CI/CD, automated testing, and version control
Requirements
What you’ll need- 7+ years of professional experience in data engineering, data architecture, or big data platforms
- Advanced knowledge of AWS Data Lakes, including experience migrating from cloud-based data platforms to lakehouse architectures
- Extensive hands-on experience building complex ingestion and transformation workflows using AWS Glue and Databricks Jobs
- Strong knowledge of Delta Lake internals, including merge/upsert patterns, table optimization (OPTIMIZE, Z-ORDER, VACUUM), and schema management at scale
- Proficiency with the Databricks platform, including PySpark, Delta Live Tables, and Serverless SQL Warehouses
- Hands-on experience configuring and managing Databricks Unity Catalog for centralized metadata management, access control, and auditing across multiple workspaces
- Experience building data platforms for ML workflows, MLOps, and conversational AI tools—specifically Databricks Genie—including curating Genie spaces, tuning prompts/instructions, and integrating with Unity Catalog semantics
- Advanced proficiency in Python and SQL
Benefits
Comp & perks- Dental Plan | Amil Dental
- Health Plan | Bradesco Saúde
- Mental Health Platform
- Telemedicine Platform
- Life Insurance | Prudential
- Pharmacy Benefit
- Gym Allowance
- Birthday Day Off
- Flexible Meal Voucher
- Home Office Allowance
- Childcare Assistance
- Transportation Voucher and Company Shuttle
- Discount Platform (Allya, SwileShop, and Dell)
- Educational Benefits
- Travel Benefits
- Pet Health Plan
- SESC Benefits
- Profit-Sharing Program (PPRL)