Itix

Data Engineer

• Design and build ETL/ELT pipelines in Databricks or using open-source strategies, ingesting and processing data within a medallion architecture on Delta Lake
• Ingest and standardize PDF documents and images with metadata
• Perform data curation and anonymization in compliance with Brazil’s LGPD, generating datasets for RAG, few-shot learning, and testing
• Build and operate vector indexing, including chunking, embedding generation, incremental index updates, and versioning
• Prepare versioned datasets (golden sets) for accuracy evaluation and benchmarking
• Structure feedback-loop persistence and feed the Quality and SLA Dashboard
• Integrate databases with inference services and APIs to other systems, taking performance, cost, monitoring, and alerting into account
• Automate processes using Workflows/Jobs, CI/CD, and data quality tests

Data Engineer – Mid-Level