Enable Data

Cloud Engineer

• Build production-ready AI applications and deploy them on Azure Databricks and Azure cloud infrastructure
• Design end-to-end data and AI pipelines using Azure Databricks
• Develop ETL/ELT workflows with Python/PySpark and SQL
• Implement CI/CD pipelines for Databricks jobs, notebooks, and workflows
• Integrate Databricks with Azure Data Lake, Blob Storage, Key Vault, Azure OpenAI, Azure Functions, and related services
• Optimize jobs for performance, cost, and reliability
• Build reusable, modular code
• Collaborate with data scientists, platform engineers, and business stakeholders to move models from experimentation to production
• Implement logging, monitoring, and error handling for production pipelines
• Develop and deploy ML and Generative AI models, including LLMs, embeddings, and RAG pipelines, for NLP, computer vision, and predictive analytics
• Fine-tune LLMs using LoRA/QLoRA and integrate them with Azure OpenAI or Hugging Face models
• Implement vector search and retrieval pipelines using FAISS or Azure Cognitive Search
• Apply responsible AI practices, including bias detection and model governance
• Ingest data into Databricks from sources such as SAP and website scraping
• Write scenario-based and edge-case tests and finalize code changes and acceptance criteria with stakeholders
• Send custom notifications from Databricks using APIs and custom Teams channel webhooks
• Build chat UI interfaces and Claude-based applications
• Use graphs to detect rule dependencies and circular references
• Reengineer code and map it to user requirements
• Implement AI safety guardrails and caching mechanisms

Lead/Sr AI Engineer, Azure Data Bricks

Data Engineer

• Modify source code for architectural considerations, including authentication changes and Databricks X-forwarded headers while using Apps.
• Participate in technical reviews for configuring Azure resources for application deployment.
• Prepare technical and network architecture documentation for cloud and security team reviews and approvals.
• Design and build Databricks data-ingestion pipelines from sources such as SAP and website scraping using PySpark and PySQL.
• Perform scenario-based and edge-case testing; collaborate with stakeholders to finalize code changes and acceptance criteria.
• Configure and use Databricks Unity Catalog permissions, OBO tokens, and service principals for Databricks Agents, Genie Spaces, Machine Learning, and Foundation Models.
• Build and deploy Databricks Apps using Visual Studio Code.
• Send custom Databricks notifications through APIs to custom Teams channel webhooks.
• Build minimal SPA chat interfaces using FastAPI, React, and Claude models/APIs.
• Apply system prompts, user prompts, prompt engineering, token economics, rule enforcement, Claude Vision API, NLP, RAG, AI safety guardrails, and caching mechanisms.
• Work with Claude Code and Genie.
• Troubleshoot full-stack frontend, backend, middleware, data-layer, and routing issues.
• Correct and rewrite code related to security findings.
• Document code using TDD, FDD, DFD, and SD techniques.
• Create and automate CI/CD deployment pipelines for Teams bots, web applications, and Databricks bundles/notebooks through ADO.
• Add validation steps such as Snyk scans and linting.
• Configure environments through CI/CD pipelines.
• Collaborate with toxicologists and client project managers from short requirement descriptions.
• Deliver work as an individual contributor in short sprint cycles.

Lead/Sr AI Engineer, Azure Data Bricks

Lead Data Engineer, CICD, AI