Firmable

Data Engineer

• Architect and own the end-to-end extraction and ETL pipeline transforming unstructured web data into a B2B dataset across 13 markets
• Set architectural standards for extractor patterns, proxy strategy, LLM infrastructure, agentic escalation workflows, coverage, schema, and accuracy
• Design extraction, normalisation, deduplication, validation, and load processes
• Own cost, performance, reliability, scheduling, incremental processing, and recovery design
• Build extractor frameworks and ship difficult extractors handling anti-bot defences, JS-heavy rendering, schema drift, and low-quality structure
• Build agentic extraction pipelines with rule-based triage, LLM escalation, structured-output validation, retries, and human-review queues
• Establish production LLM infrastructure with versioned prompts, labelled evaluation sets, precision/recall measurement, rollback, traces, and drift detection
• Develop evaluation and observability scaffolding and model-choice playbooks
• Create versioned SKILL.md specifications and orchestration patterns using Airflow or equivalent
• Provide technical input to data platform, product, and analytics teams
• Deliver approximately 80% hands-on architecture/reference implementation and 20% cross-functional technical input

Lead Data Engineer