FREE ACCESS
5,000–10,000 jobs/day
See all jobs on Scoutfield
Search thousands of fresh jobs every day.
Discover
- Fresh listings
- Fast filters
- No subscription required
Create a free account and start exploring right away.
Core Competencies
Role fitCore Competencies
Use this summary to align your resume positioning with the role.
Demonstrates expertise in designing and managing Databricks workspaces, clusters, and data governance through Unity Catalog. Proficient in implementing Delta Lake and Delta Live Tables for efficient data processing and analytics in cloud environments.
Highest-signal resume keywords
Databricks Platform ExperienceDelta Lake InternalsUnity Catalog ManagementBig Data EngineeringCloud Data Platforms
ATS Keywords
Tailor your resumeApplicant Tracking System Keywords
Tip: use these terms in your resume and cover letter to boost ATS matches.
Hard Skills
Data EngineeringDelta Live TablesDatabricks WorkspacesCluster OptimizationSQL AnalyticsData GovernancePipeline DevelopmentChange Data FeedMedallion ArchitectureTransaction Monitoring
Tools & Technologies
DatabricksAzure Key VaultAWS Secrets ManagerKafkaADLSS3GCSDatabricks SQL WarehousesDatabricks PhotonCost Optimization Strategies
Certifications & Qualifications
Bachelor's Degree in Computer ScienceMaster's Degree in Information TechnologyMaster's Degree in Data Engineering
Industry Keywords
Financial CrimeAMLKYCCRASanctions ScreeningFraud DetectionRegulatory ReportingData LineageAccess ControlDistributed Computing
Tech Stack
Tools & technologiesAWSAzureCloudETLKafkaSQLUnityVault
About the role
Key responsibilities & impact- Design, build, and maintain Databricks workspaces, clusters, and compute pools across dev/test/prod environments
- Configure and manage Databricks Unity Catalog for data governance, access control, fine-grained permissions, and data lineage
- Optimize cluster configurations, including instance types, auto-scaling policies, and spot/preemptible nodes, for cost and performance
- Implement workspace-level best practices, including folder structures, access controls, and secret management with Databricks Secrets, Azure Key Vault, or AWS Secrets Manager
- Manage Databricks jobs, workflows, and multi-task job orchestration with dependency management
- Design and implement Delta Lake tables with partitioning, Z-ordering, and file compaction using OPTIMIZE and VACUUM
- Build Medallion Architecture Bronze, Silver, and Gold layers for structured data lake organization
- Implement Delta Live Tables pipelines for declarative, reliable ETL/ELT with built-in data quality expectations
- Manage schema evolution, table versioning, time travel, and Change Data Feed for incremental processing
- Design data lakehouse patterns integrating Delta Lake with Kafka, ADLS, S3, and GCS
- Create Databricks pipelines delivering Financial Crime platforms covering AML, KYC, CRA, Sanctions Screening, Transaction Monitoring, Fraud Detection, and Regulatory Reporting
Requirements
What you’ll need- Bachelor's or Master's degree in Computer Science, Information Technology, Data Engineering, or related field
- 8+ years of total experience in data engineering or software engineering
- 3+ years of dedicated hands-on experience with the Databricks platform in production environments
- Strong background in big data engineering, cloud data platforms, and distributed computing
- Deep expertise in Databricks Workspaces, Clusters, Jobs, Workflows, and Repos
- Proficiency with Unity Catalog, including metastore setup, catalog/schema/table management, access controls, and data lineage
- Hands-on experience with Delta Live Tables (DLT), including pipeline development, expectations, and monitoring
- Strong command of Delta Lake internals, including transaction log, ACID guarantees, file layout, and optimization techniques
- Experience with Databricks SQL Warehouses, SQL Analytics, and dashboard creation
- Knowledge of Databricks Photon engine, serverless compute, and cost optimization strategies