Data Engineering Manager – GCP, AI/ML, GenAI

Naveera Technology LLC

. Lead the end-to-end migration of enterprise data platforms from AWS to GCP .

Posted 10/7/2026full-timeRemote • United StatesSeniorLeadWebsite

Tech Stack

Tools & technologies

AirflowAmazon RedshiftApacheAWSAzureBigQueryCloudDockerETLFlaskGoogle Cloud PlatformKafkaKubernetesPySparkPythonPyTorchScikit-LearnSparkSQLTensorflowTerraform

About the role

Key responsibilities & impact

Lead the end-to-end migration of enterprise data platforms from AWS to GCP
Architect and implement scalable enterprise data platforms, Data Lake, and Lakehouse architectures on GCP
Design and implement AI/ML, Generative AI, RAG, and MLOps solutions using GCP and Vertex AI
Build production-grade machine learning pipelines covering preparation, training, validation, deployment, monitoring, retraining, and lifecycle management
Design ingestion, transformation, chunking, embedding, indexing, retrieval, batch, streaming, ETL, and ELT pipelines
Analyze AWS platforms and map AWS workloads and services to equivalent or improved GCP services
Define migration roadmaps, phases, dependencies, risks, rollback strategies, and target-state architectures
Design real-time event-driven pipelines using Pub/Sub, Dataflow/Apache Beam, BigQuery, and Cloud Storage
Define enterprise data models, BigQuery partitioning and clustering strategies, and analytics consumption models
Establish data governance, quality, lineage, metadata, security, privacy, access control, and responsible AI standards
Lead Terraform infrastructure automation, CI/CD, deployment, testing, and environment promotion across Dev, QA, UAT, and Production
Optimize performance, scalability, latency, throughput, reliability, and cloud costs; establish benchmarks and SLAs
Lead and mentor Data Engineers, Senior Data Engineers, and Technical Leads; conduct architecture and code reviews
Track engineering progress, risks, dependencies, and delivery milestones
Serve as primary technical contact for US-based stakeholders and collaborate with Business, Product, Data Science, BI, DevOps, Security, and Analytics teams
Translate business requirements into technical solutions and present architecture decisions, migration strategies, roadmaps, risks, and trade-offs

Requirements

What you’ll need

15+ years of experience in GCP Data Engineering, Data Architecture, Cloud Engineering, AI/ML Engineering, or related technology leadership roles
5+ years of strong hands-on GCP Data Engineering experience
3+ years of strong hands-on AI/ML and GenAI experience
Proven experience delivering AWS-to-GCP migration projects
Strong experience designing enterprise Data Lake and Lakehouse platforms on GCP
Strong hands-on experience with BigQuery, Google Cloud Storage, Dataflow, Pub/Sub, Cloud Composer, Dataproc, IAM, and Terraform
Experience migrating AWS data workloads, pipelines, and platforms to GCP
Strong knowledge of AWS and GCP service mapping, migration patterns, modernization strategies, and cloud architecture best practices
Experience designing, building, and deploying AI/ML solutions on GCP using Vertex AI
Hands-on experience with Generative AI, LLM-based applications, RAG architectures, embeddings, vector search, prompt engineering, and enterprise AI assistants
Strong understanding of MLOps, including model training, model registry, CI/CD/CT, model deployment, monitoring, retraining, governance, and rollback strategies
Experience implementing secure and responsible AI solutions, including data privacy, model evaluation, access controls, auditability, and governance
Expert-level SQL and strong Python and PySpark skills
Strong data modeling, data warehousing, batch processing, and real-time data engineering experience
Experience with Terraform, Git, GitHub, Cloud Build, CI/CD pipelines, and infrastructure automation
Experience managing and mentoring data engineering and cross-functional technical teams
Strong communication skills with experience working with US-based stakeholders
Google Cloud Professional Data Engineer certification preferred
Google Cloud Professional Machine Learning Engineer certification preferred
Experience with Vertex AI Agent Builder, Vertex AI Search, Gemini models on Vertex AI, or enterprise Generative AI platforms preferred
Experience with dbt, Apache Airflow, Kafka, Apache Spark, Kubernetes, Cloud Run, and API-driven architectures preferred
Experience with Dataplex, Data Catalog, data lineage, metadata management, data governance, master data management, and data-quality frameworks preferred
Experience supporting enterprise or regulated environments with strong data privacy, security, compliance, audit, and governance requirements preferred
Required technical skills include Python, PyTorch, TensorFlow, Scikit-learn, NLP, deep learning, ML algorithms, GenAI, LLMs, GPT, Gemini, Claude, Llama, prompt engineering, fine-tuning, RAG, embeddings, vector databases, semantic search, hybrid search, reranking, LangChain, LlamaIndex, LangGraph, Hugging Face, Transformers, AI agents, agentic workflows, tool/function calling, multi-agent systems, MCP, MLflow, Kubeflow, model registry, model deployment, monitoring, CI/CD, Vertex AI, Vertex AI Studio, Vertex AI Pipelines, Model Garden, Vector Search, FastAPI, Flask, REST APIs, SQL, Docker, Kubernetes, GCP, AWS, Azure, BigQuery, Dataflow, Spark, Databricks, data lakes, advanced Python, expert SQL, PySpark, Apache Spark, ETL, ELT, CDC, batch and streaming, event-driven architecture, data pipeline development and optimization, enterprise Data Lake/Lakehouse, Medallion Architecture, data warehousing, data modeling, dimensional modeling, multi-tenant modeling, schema-on-read/schema-on-write, dbt, Apache Airflow/Cloud Composer, Dataproc, Dataflow/Apache Beam, AWS Glue, Redshift, EMR, Lambda, Kinesis, Athena, CloudWatch, GCS, AWS S3, Pub/Sub, Dataplex, Data Catalog, Terraform, Git/GitHub, Cloud Build, CI/CD, Infrastructure as Code, data lineage, metadata management, data quality, monitoring, and OpenLineage

Benefits

Comp & perks

Flexible remote work
Exposure to global customers
Collaborative, innovation-driven culture
Continuous learning and certification
Opportunity to lead transformative AI/ML and GCP innovations as Head of Engineering