Data Scientist, Applied AI

Definitive Healthcare

. Implement and test LLM, RAG, and agent-based workflows for assigned use cases under senior team direction .

Posted 10/27/2026full-timeFramingham • Massachusetts • United StatesMid-LevelSenior💰 $111,000 - $167,000 per yearWebsite

Core Competencies

Role fit

Core Competencies

Use this summary to align your resume positioning with the role.

Demonstrates expertise in implementing and testing LLMs, retrieval-augmented generation, and agent-based workflows, with strong capabilities in data analysis and problem-solving. Proficient in developing AI workflows, evaluating model performance, and collaborating effectively with cross-functional teams.

Highest-signal resume keywords

LLM DevelopmentPython ProgrammingData AnalysisRAG Pipeline DevelopmentAgent Framework Experience

ATS Keywords

Tailor your resume

Applicant Tracking System Keywords

Tip: use these terms in your resume and cover letter to boost ATS matches.

Hard Skills

Data ScienceMachine LearningApplied AIEmbeddingsSemantic SearchRetrievalEvaluation MetricsTest Case DesignPerformance MonitoringExperimentation

Soft Skills

Clear CommunicationProblem-Solving

Tools & Technologies

PydanticAILangGraphDatabricksCloud Data PlatformsGit

Industry Keywords

AI WorkflowsRetrieval-Augmented GenerationBenchmark DatasetsRegression ChecksProduction Deployment

Tech Stack

Tools & technologies

CloudPython

About the role

Key responsibilities & impact

Implement and test LLM, RAG, and agent-based workflows for assigned use cases under senior team direction
Build and test AI workflows using LLMs, retrieval-augmented generation, and tool-using agents
Compare LLMs, embedding models, and retrieval methods for quality, reliability, latency, and cost
Create and update representative test cases, benchmark datasets, and regression checks
Report evaluation results and limitations
Develop and tune RAG pipelines, including embeddings, retrieval, reranking, and grounding
Investigate incorrect answers, retrieval misses, and workflow failures
Implement and measure improvements
Use experimentation, tracing, and observability tools to monitor systems and investigate performance changes
Collaborate with engineering, product, and domain partners to clarify requirements, communicate findings, and deliver practical AI solutions
Work independently on defined analyses and implementation tasks
Recommend model and retrieval improvements and escalate major architecture, sensitive-data, cost, and production-release decisions to senior team members

Requirements

What you’ll need

Bachelor’s degree in a quantitative or technical field
3+ years of experience in data science, machine learning, applied AI, or a related field
Hands-on experience developing applications with LLMs and evaluating their outputs
Strong Python, data analysis, and problem-solving skills
Understanding of embeddings, semantic search, retrieval, RAG, and evaluation metrics
Experience with an agent framework such as PydanticAI, LangGraph, or a similar tool
Ability to design useful test cases, interpret evaluation results, and identify limitations
Familiarity with cloud data or ML platforms, Git, testing, and collaborative software development
Clear communication with technical and nontechnical colleagues
Experience with Databricks, evaluation platforms, production deployment, fine-tuning, distributed processing, or healthcare data is beneficial but not required

Benefits

Comp & perks

Company bonus or commission plan eligibility, depending on the position
Medical, dental, and vision coverage
Unlimited paid time off
401(k) plan with employer contribution
Industry leading products
Flexible and dynamic culture
Limitless opportunity through fast growth
Competitive benefits package including great healthcare benefits and a 401(k) match
Definitive Cares community service group
Work-life balance