Data Engineer

Rain

. Build scalable batch and streaming pipelines, selecting approaches based on latency, cost, and correctness .

Posted 11/12/2026full-timeUnited StatesMid-LevelSenior💰 $128,393 - $167,498 per yearWebsite

Core Competencies

Role fit

Core Competencies

Use this summary to align your resume positioning with the role.

Demonstrates expertise in building scalable batch and streaming data pipelines, utilizing SQL and Python/Rust for production data solutions. Proficient in data quality testing, monitoring, and reconciliation, with experience in cloud data warehouses and orchestration tools.

Highest-signal resume keywords

Batch And Streaming Pipeline DevelopmentSQL And Python/Rust ProficiencyCloud Data Warehouse ExperienceData Quality Testing And MonitoringPipeline Orchestration With Dagster

ATS Keywords

Tailor your resume

Applicant Tracking System Keywords

Tip: use these terms in your resume and cover letter to boost ATS matches.

Hard Skills

Batch ProcessingStreaming ProcessingSQLPythonRustData Quality TestingMonitoringDebuggingData ModelingData Reconciliation

Tools & Technologies

BigQueryClickHouseSnowflakePub/SubKafkaDataflow/BeamFlinkDbtDagsterAirflow

Industry Keywords

Data SolutionsBatch IngestionCDCSFTPGCSMachine Learning FeaturesPaymentsBankingBlockchain DataEVM

Tech Stack

Tools & technologies

AirflowBigQueryCloudGoogle Cloud PlatformKafkaPythonRustSnowflakeSQL

About the role

Key responsibilities & impact

Build scalable batch and streaming pipelines, selecting approaches based on latency, cost, and correctness
Process near-real-time events with deduplication, windowing, and handling of late or out-of-order events
Own batch ingestion from databases using CDC, files via SFTP and GCS, APIs, and onchain sources
Build incremental dbt models handling late data, duplicates, CDC deletes, and backfills
Reconcile streaming and batch outputs
Orchestrate pipelines in Dagster with sensors, schedules, and alerts
Add freshness, volume, and reconciliation checks to every source
Fix root causes of data quality issues
Work with backend pods on event and table contracts to prevent downstream breakage
Partner with Analytics Engineers, Data Scientists, and Risk on data needs, including ML features
Automate ingestion and quality checks so new sources come online quickly and remain trustworthy

Requirements

What you’ll need

At least 3 years building production data solutions, owning batch and streaming pipelines end to end
Experience designing data solutions at scale
Strong SQL and Python / Rust
Experience with a cloud data warehouse such as BigQuery, ClickHouse, or Snowflake, including understanding of trade-offs
Streaming experience with Pub/Sub, Kafka, Dataflow/Beam, or Flink
Batch experience with dbt and Dagster or Airflow
Experience with production data quality testing, monitoring, and reconciliation
Ability to debug across the source database, pipeline, warehouse, and report
Ability to use AI tools to learn and work faster while checking their output
Ability to work in a hybrid role and come into a NYC office 3 days a week
Legally authorized to work in the United States for any employer without restrictions
Nice to have: offline and real-time ML feature pipelines
Nice to have: payments, banking, or blockchain data experience, including EVM or Solana events and indexers
Nice to have: CDC and ingestion tools such as Datastream, Debezium, Meltano, Fivetran, or Airbyte
Nice to have: experience with GCP

Benefits

Comp & perks

Flexible time off with a 10-day minimum vacation requirement
Flexible working: work from home, in an office, or both
Home workspace setup stipend for new Rainmakers
For US Rainmakers, 95% coverage of health, dental, and vision plan costs
90% coverage of health, dental, and vision plan costs for dependents
100% company-subsidized life insurance for US Rainmakers
401(k) with a 4% company match
Equity option plan
Monthly health and wellness stipend
DoorDash credit for in-office lunch and dinner
Domestic and international team and company offsites