Clinical AI Engineer

Heidi Health

. Join Clinical AI R&D as a Clinical AI Engineer inside a product pod .

Posted 10/28/2026full-timeMelbourne • AustraliaMid-LevelSeniorWebsite

Core Competencies

Role fit

Core Competencies

Use this summary to align your resume positioning with the role.

Demonstrates expertise in clinical AI evaluation, including study design, statistical analysis, and the ability to interpret clinical safety findings. Proficient in Python or TypeScript for code modifications and has a strong background in peer-reviewed research related to clinical AI.

Highest-signal resume keywords

Medical DegreeIndependent Clinical JudgmentStudy DesignInferential StatisticsLLM Evaluation

ATS Keywords

Tailor your resume

Applicant Tracking System Keywords

Tip: use these terms in your resume and cover letter to boost ATS matches.

Hard Skills

Statistical AnalysisPrompt EngineeringMachine LearningEvaluation Report WritingClinical AI Evaluation

Soft Skills

Clear Writing SkillsAbility to Work Independently

Tools & Technologies

PythonTypeScript

Industry Keywords

Post-Market SurveillanceClinical SafetyMedical Device FrameworkPeer-Reviewed PublicationsLLM-as-Judge Pipelines

Tech Stack

Tools & technologies

PythonTypeScript

About the role

Key responsibilities & impact

Join Clinical AI R&D as a Clinical AI Engineer inside a product pod
Own the evidence for the product throughout its lifecycle
Define evaluations for greenfield products, including endpoints, comparators, ground truth, sample size and analysis plans
Build, optimize and validate LLM judges and rubrics at scale
Report agreement between LLM judges and clinician ground truth
Execute evaluations, perform statistical analysis and prepare reports for the product pod, regulatory files and academic submissions
Monitor output quality in production through post-market surveillance
Respond to regulatory feedback with data and interpret clinical safety findings for the product
Change prompts, rubrics, evaluation harnesses and production code based on evidence
Contribute to peer-reviewed literature on the product’s translational benefit
Provide clinical judgment and identify clinically incorrect model outputs
Work with pod engineers and clinicians through normal code review and release processes

Requirements

What you’ll need

A medical degree and clinical practice with independent clinical judgment
Study design, inferential statistics, agreement and reliability measures, sample size, and handling of multiple comparisons
Thesis or publication-level academic work
Experience evaluating medical software or clinical AI, including building or validating LLM-as-judge pipelines
Ability to read and understand a production codebase, assess the impact of changes, and make changes in Python or TypeScript
Prompt engineering and AI-assisted coding experience
Ability to work independently with light direction
Clear writing skills for evaluation reports, surveillance summaries and manuscripts
Post-market surveillance or vigilance work under a medical device framework is nice to have
Peer-reviewed publications on clinical AI or LLM evaluation are nice to have
Machine learning background beyond prompting is nice to have
Specialty depth in an area Heidi is moving into is nice to have
Academic or clinical-site network able to host a study is nice to have

Benefits

Comp & perks

Equity from day one
Comprehensive private medical and dental cover through Bupa
24/7 mental health, coaching and wellbeing support through Sonder
£100/month Healthy Heidi’s stipend
26 weeks paid parental leave for primary carers and 18 weeks for secondary carers, subject to eligibility
£7,000 one-off fertility support payment, subject to eligibility
£700 per year for courses, books, memberships, conferences and more
£500 one-off home office setup budget
Recharge days after major milestones and busy periods
Work from anywhere for up to 4 weeks per year
10 days per year clinical leave for eligible clinical roles
Hybrid environment with 3 days in the office