Lead Data Engineer

Capital Technology Group, LLC

. Design, build, and maintain scalable data pipelines, ETL/ELT workflows, and data models .

Posted 9/24/2026full-timeRemote • United StatesSenior💰 $150,000 - $200,000 per yearWebsite

Core Competencies

Role fit

Core Competencies

Use this summary to align your resume positioning with the role.

Demonstrates expertise in designing and optimizing AWS-native data platforms, building scalable data pipelines, and integrating diverse data sources. Proficient in leading modernization initiatives and mentoring teams in an Agile environment while ensuring compliance with federal data standards.

Highest-signal resume keywords

AWS GlueApache Spark (PySpark)Data Pipeline DevelopmentETL/ELT ArchitecturePublic Trust Clearance

ATS Keywords

Tailor your resume

Applicant Tracking System Keywords

Tip: use these terms in your resume and cover letter to boost ATS matches.

Hard Skills

PythonSQL (PostgreSQL)Apache AirflowAmazon RedshiftAmazon S3AWS LambdaAWS Step FunctionsAmazon EMRData Integration SolutionsPerformance Tuning

Soft Skills

Analytical SkillsProblem-Solving SkillsCommunication Skills

Tools & Technologies

CloudFormationGitHubHarnessCI/CD PipelinesAmazon CloudWatchAmazon DMSAmazon MWAAEventBridgeOpenSearchTrino

Industry Keywords

FedRAMPNIST 800-53Agile EnvironmentData EngineeringNoSQL Platforms

Tech Stack

Tools & technologies

AirflowAmazon RedshiftApacheAWSCloudETLHadoopJavaNoSQLOraclePostgresPySparkPythonSparkSQL

About the role

Key responsibilities & impact

Design, build, and maintain scalable data pipelines, ETL/ELT workflows, and data models
Develop and optimize AWS-native data platforms using AWS Glue, Amazon EMR, Amazon MWAA (Apache Airflow), Lambda, Step Functions, Amazon S3, Redshift, RDS, DMS, and CloudWatch
Build high-performance ingestion, transformation, and orchestration workflows for structured and semi-structured data
Design and optimize analytical data platforms using Amazon Athena, Trino, Hive, OpenSearch, and enterprise data catalog technologies
Integrate enterprise and external data sources across relational and NoSQL platforms
Build AI-enabled data solutions using Amazon Bedrock, RAG pipelines, and vector search technologies
Develop cloud infrastructure using CloudFormation, GitHub, Harness, CI/CD pipelines, SNS, SQS, and EventBridge
Improve reliability, scalability, performance, and maintainability through monitoring, troubleshooting, automation, and continuous optimization
Support mission-critical analytics and reporting solutions in AWS-based federal data environments complying with FedRAMP and NIST 800-53 controls
Lead modernization initiatives migrating IBM DataStage, Hadoop, RunDeck, and shell-based workflows to cloud-native AWS services
Mentor junior engineers through technical guidance, architecture discussions, and code reviews
Collaborate with cross-functional teams in an Agile environment to define requirements, deliver data solutions, and communicate technical concepts

Requirements

What you’ll need

Applicants must be US Citizens and be able to obtain Public Trust clearance
Bachelor's degree in Computer Science, Engineering, or a related technical field
15+ years of professional experience in data engineering or related domains
Strong hands-on experience with Apache Spark (PySpark), Python, SQL (PostgreSQL), and dbt
Strong hands-on experience with AWS Glue, Amazon EMR, Amazon MWAA (Apache Airflow), AWS Lambda, AWS Step Functions, Amazon S3, Amazon Redshift, Amazon RDS, AWS DMS, and Amazon CloudWatch
Experience developing scalable data pipelines, workflow orchestration, and data integration solutions across enterprise environments
Experience with Apache Iceberg, Parquet, ORC, and Avro
Experience designing and optimizing solutions using PostgreSQL, Redshift, Oracle, GraphDB, and NoSQL platforms
Experience with performance tuning, system optimization, and enterprise-scale ETL/ELT architectures
Java development and modern CI/CD practices using Harness
Strong analytical and problem-solving skills
Experience working in agile, iterative software development environments
Ability to quickly learn and apply new technologies and domain knowledge
Excellent written and verbal communication skills, with the ability to explain complex topics to diverse audiences

Benefits

Comp & perks

Remote Work (Hybrid roles will be specified in the job post)
Competitive Compensation Package
Medical, Dental, and Vision
Life Insurance, Short/Long Term Disability
Employee Assistance Program
401(k) with 4% matching
Liberal PTO vacation policy
Generous Annual Continuing Education
Annual Wellness Budget
Bonus Incentive Programs (Employee referrals and performance-based rewards)