Data Engineer

• Design and deliver batch and streaming data pipelines ingesting data from SurveyCTO, ArcGIS, and custom mobile apps into RTV's Databricks warehouse
• Implement ELT and ETL workflows in PySpark and Databricks SQL using Bronze, Silver, and Gold Medallion Architecture layers
• Own pipeline workstreams from design through deployment and monitoring
• Build and maintain Delta Lake tables with schema enforcement, ACID-compliant writes, and time-travel capabilities
• Contribute to star schema design, slowly changing dimension patterns, and denormalization decisions
• Support Unity Catalog governance, lineage tracking, access controls, and dataset documentation
• Maintain data observability frameworks covering freshness, volume anomalies, schema drift, and SLA monitoring
• Build validation and quality checks using Great Expectations, dbt tests, or Databricks-native monitoring
• Contribute to structured logging, alerting, and incident response practices
• Integrate household data, image classification outputs, and ML predictions into unified warehouse layers
• Collaborate with ML Engineers and Data Scientists on feature engineering and training data preparation workflows
• Collaborate on roadmap prioritization, architectural decisions, and engineering standards
• Partner with Software Engineers, Data Scientists, field evaluation teams, and program staff to translate data requirements into pipeline solutions
• Maintain documentation covering pipeline architectures, transformation logic, data dictionaries, and runbooks