Senior Data Engineer

proSapient

. Build and operate batch and real-time pipelines feeding matching and search systems.

Posted 11/11/2026full-timeRemote • United Kingdom, PortugalSeniorWebsite

Core Competencies

Role fit

Core Competencies

Use this summary to align your resume positioning with the role.

Demonstrates expertise in building and operating production data pipelines, with strong capabilities in Python programming, SQL, and Elasticsearch. Proficient in data modeling, monitoring, and optimizing data processing for search and ranking systems.

Highest-signal resume keywords

Production Data Pipeline DevelopmentPython ProgrammingElasticsearch Performance OptimisationSQL ProficiencyData Modelling for Analytics

ATS Keywords

Tailor your resume

Applicant Tracking System Keywords

Tip: use these terms in your resume and cover letter to boost ATS matches.

Hard Skills

PythonSQLElasticsearchData ModellingBatch ProcessingEvent StreamingKafkaPostgreSQLBigQueryCI/CD

Tools & Technologies

DockerKubernetesAWSGCPPrometheusDatadogRabbitMQDbt

Industry Keywords

Data PipelinesSearch SystemsMachine LearningData QualityObservability

Tech Stack

Tools & technologies

AWSBigQueryCloudDockerElasticSearchElixirGoogle Cloud PlatformKafkaKubernetesPostgresPrometheusPythonRabbitMQSQL

About the role

Key responsibilities & impact

Build and operate batch and real-time pipelines feeding matching and search systems.
Work with real-time data streams and search infrastructure using Kafka and Elasticsearch.
Model datasets consumed by the product layer, including candidate features, interaction events and ranking signals, while maintaining stable contracts.
Own production pipelines, including monitoring, alerting, backfills and on-call support.
Instrument recruiter and client behaviour so matching quality can be measured and improved.
Build event pipelines supporting CSAT and workflow metrics.
Conduct and analyse experiments to iterate on data-processing pipeline quality and make experiment data queryable.
Add data-quality checks and tests at important boundaries and make failures visible.
Reduce relevant cost and latency.
Partner with QA to make data correctness testable.
Identify and provide useful internal or external data sources for structured search and GenAI tools.
Work directly with the Lead Search Engineer, AI/ML Engineer and Product Manager on a frequently shipping team.

Requirements

What you’ll need

5+ years building production data pipelines.
Strong Python programming skills, with practical experience writing clean, production-ready code.
Strong SQL and proficiency with PostgreSQL or other relational databases.
Strong hands-on experience with Elasticsearch / OpenSearch, including performance optimisation and query tuning.
Deep understanding of both batch and event/streaming patterns, and a clear sense of when each is appropriate.
Practical data modelling skills for analytics and for serving application features.
Experience operating your own pipelines in production, alongside comfort working in a codebase with tests, code review and CI.
Message brokers like Kafka or RabbitMQ (nice to have).
Cloud data warehouse experience, with BigQuery preferred and equivalents acceptable (nice to have).
Docker, Kubernetes, and cloud environments such as AWS and/or GCP (nice to have).
Experience feeding search, ranking or ML systems specifically (nice to have).
Building LLM-based extraction or enrichment pipelines (nice to have).
Experience with dbt or a comparable transformation framework (nice to have).
Observability tools such as Prometheus or Datadog and background job processing frameworks (nice to have).
Exposure to Elixir services (nice to have).

Benefits

Comp & perks

Tenure gifts, including vouchers, extra holiday and sabbaticals for each year of employment.
Health insurance through Vitality.
Remote working for up to 20 days each year, giving you flexibility and a change of scenery.
Employee Assistance Programme with personalised health and wellbeing advice from specialist teams.
Enhanced maternity and paternity pay.
25 days’ annual leave plus bank holidays, including a week’s closure over Christmas.
MyMindPal app for online mental fitness support.
Corporate events, from quarterly gatherings to annual winter and summer parties.