Scientist

• Design, build, and maintain automated evaluation pipelines for LLM quality, RAG performance, agent reliability, safety, and scientific accuracy
• Author evaluation rubrics and scoring criteria
• Curate golden and synthetic datasets with domain experts
• Maintain a registry of reusable evaluation assets
• Validate AI judges against human expert agreement
• Run model, prompt, retriever, and agent benchmarks producing standardized quality readouts
• Analyze failure patterns such as hallucination, unsupported claims, and weak traceability
• Turn evaluation findings into actionable recommendations
• Develop therapeutic-area-specific evaluation criteria with scientific, clinical, and regulatory partners
• Design evaluation methods for scientific reasoning, evidence synthesis, and hypothesis quality
• Build evaluation tooling and reusable patterns enabling other teams to self-serve
• Inform AI release decisions for pharmaceutical R&D workflows

Scientist jobs in Spain

Senior Scientist – GenAI Evaluation