Artificial Intelligence

• Perform model validations on GenAI and agentic AI use cases across functional areas and lines of business
• Ensure model calculations, algorithms, and methods are accurate and appropriate for intended use
• Design and build challenger solutions and testing methods for summarization, question answering, search, data synthesis, LLM-as-a-judge, and related tasks
• Review quantitative and qualitative testing techniques for model accuracy, robustness, and reliability
• Assess data inputs, assumptions, prompt engineering, and context engineering
• Design and execute testing approaches for non-deterministic AI systems, including variability analysis, confidence intervals, and statistical evaluation methodologies
• Challenge modeling elements including inputs, calculations, outputs, conceptual soundness, monitoring and controls, and documentation
• Identify findings and recommendations, perform impact analysis, and compile model validation reports
• Perform governance accountabilities related to findings tracking, remediation testing, and validation
• Enhance the GenAI model validation framework with standardized evaluation metrics and validation tools
• Partner with Data Science teams to align model risk practices with evolving modeling capabilities and promote proactive risk management
• Stay informed about AI/ML, GenAI, agentic systems, regulatory expectations, and department initiatives
• Assist with evaluating and testing tools including VertexAI/Google ADK, LangChain/LangGraph, RAG frameworks, HuggingFace, and OpenAI APIs
• Assess solution alignment with AI Governance standards, Model Risk Management policies, and emerging regulatory expectations

Explore Senior Analyst, Model Risk & AI – Gen AI Model Validation at The Hartford. Review requirements and application details.