Senior AI Data Engineer

Cadre AI

. Design and build medallion lakehouses on Databricks, Microsoft Fabric, and Snowflake .

Posted 11/6/2026full-timeRemote • ColombiaSenior💰 COP 6,000 - COP 7,500 per yearWebsite

Core Competencies

Role fit

Core Competencies

Use this summary to align your resume positioning with the role.

Demonstrates expertise in designing and operating production lakehouses on Databricks and Microsoft Fabric, with a strong focus on data pipeline orchestration, retrieval architectures, and data governance. Proficient in SQL, Python, and dbt, with a solid understanding of machine learning principles and data quality management.

Highest-signal resume keywords

Databricks Lakehouse DesignData Pipeline OrchestrationExpert SQLProduction RAG Pipeline ExperienceGraphRAG Retrieval Design

ATS Keywords

Tailor your resume

Applicant Tracking System Keywords

Tip: use these terms in your resume and cover letter to boost ATS matches.

Hard Skills

SQLPythonPySparkDbtApache AirflowDagsterData Quality ChecksChange Data CaptureHybrid RetrievalData Governance

Soft Skills

Client-Facing ConsultingCollaborationDocumentation

Tools & Technologies

DatabricksMicrosoft FabricSnowflakeGitHub ActionsMosaic AI Agent FrameworkLangfuseMLflowPower BICubeAgent Bricks

Certifications & Qualifications

Master’s DegreePhD in Computer ScienceMathematicsEngineering

Industry Keywords

Medallion ArchitectureKnowledge GraphsRBACSOC 2 ComplianceAI Governance

Tech Stack

Tools & technologies

AirflowApachePySparkPythonSnowflakeSQLUnity

About the role

Key responsibilities & impact

Design and build medallion lakehouses on Databricks, Microsoft Fabric, and Snowflake
Own ingestion from ERPs, CRMs, and SaaS platforms, including change data capture and schema drift handling
Model and transform data with dbt and orchestrate pipelines with Airflow, Dagster, or Lakeflow Jobs
Ship pipelines through CI/CD using GitHub Actions and Declarative Automation Bundles
Build data quality checks and data contracts into pipelines
Configure and govern coding agents in client Databricks repositories
Build production RAG pipelines including parsing, OCR, chunking, embeddings, hybrid retrieval, and reranking
Design GraphRAG systems using knowledge graphs and graph stores
Select and justify retrieval architectures based on use case, cost, and maintenance
Operate vector, full-text, and hybrid search infrastructure with incremental indexing and permission-aware retrieval
Build and deploy retrieval-backed agents on Databricks Agent Bricks
Create governed semantic and metric definitions across dbt, Power BI, Cube, Databricks, and Snowflake
Lead data readiness assessments and document business logic, gaps, and remediation
Expose governed data to agents through MCP
Build golden datasets, evaluation pairs, and leakage-free backtesting splits
Instrument retrieval and agent pipelines and track recall, precision, groundedness, experiments, and model versions
Monitor drift, freshness, and production data gaps
Implement RBAC, row-level security, and masking across data and retrieval systems
Ensure data paths meet SOC 2 and client AI governance requirements
Work with client IT, finance, and operations leaders on scope, access, and build-versus-buy decisions
Record architecture decisions and contribute reusable engineering blocks to Cadre’s internal library

Requirements

What you’ll need

5+ years of data engineering experience
At least one production lakehouse designed and operated on Databricks or Microsoft Fabric with medallion architecture
Master’s degree or PhD in Computer Science, Mathematics, Engineering, or related quantitative field
Production RAG pipeline experience, including hybrid retrieval, reranking, chunking, and permission-aware indexing
Experience designing and running GraphRAG or knowledge-graph retrieval
Production data-pipeline orchestration experience with Apache Airflow or Dagster
Expert SQL and strong Python and PySpark
Production experience with dbt or platform-native pipelines and ingestion tools
Daily use and review of coding agents such as Claude Code
Experience authoring semantic or metrics layers
Working ML fluency, including data leakage, time-based validation, class imbalance, precision and recall
Experience building datasets for model or retriever training/evaluation
Experience with Langfuse, Arize Phoenix, Braintrust, or MLflow and MLflow Registry
Hands-on experience building Databricks agents using Agent Bricks or Mosaic AI Agent Framework
Experience running Claude Code or Codex under governed harnesses
Familiarity with Omnigent, Genie Migrate, Agent Bricks CLI, RAG, text-to-SQL, and MCP
Governance experience across Unity Catalog, Purview, or Snowflake RBAC
Client-facing consulting or forward-deployed experience
Professional English fluency
Strong plus: Snowflake Cortex, Palantir Foundry ontologies, RDF/OWL knowledge graphs, DSPy prompt optimization, or LoRA fine-tuning

Benefits

Comp & perks

AI-native culture
Access to frontier models and capabilities through partnerships with Anthropic and OpenAI
Growth opportunities and room to move into new teams and initiatives
No bureaucracy; clear accountability and idea-driven culture