AI Engineer

CreditorWatch

. Design, build and operate production-grade AI products, APIs, RAG solutions and agentic workflows .

Posted 11/3/2026full-timeSydney • AustraliaMid-LevelSeniorWebsite

Core Competencies

Role fit

Core Competencies

Use this summary to align your resume positioning with the role.

Demonstrates expertise in designing and operating production-grade AI systems, with a strong focus on LLM architecture, model orchestration, and AI evaluation. Proven ability to mentor engineers, influence technical decisions, and establish best practices for scalable AI solutions.

Highest-signal resume keywords

Production-Grade AI Systems DesignLLM ArchitectureAI Evaluation and ObservabilityMentoring and Technical DirectionReusable Frameworks and Tooling

ATS Keywords

Tailor your resume

Applicant Tracking System Keywords

Tip: use these terms in your resume and cover letter to boost ATS matches.

Hard Skills

Software EngineeringSystem DesignOperational DisciplineModel OrchestrationAgentic WorkflowsAI Safety and GovernanceAI Testing and SecurityBuilding Scalable SolutionsTechnical Trade-OffsCross-Team Engineering Initiatives

Soft Skills

Clear CommunicationNavigating AmbiguityOwnership of OutcomesBuilding Cross-Functional Relationships

Tools & Technologies

LangfuseMLflowOpenTelemetryDatadogPostgreSQLOpenSearchVector DatabasesAWSDockerGitHub

Industry Keywords

AI ProductsRAG SolutionsAgentic SystemsProduction ReliabilityRisk-Sensitive Environments

Tech Stack

Tools & technologies

AWSDockerPostgres

About the role

Key responsibilities & impact

Design, build and operate production-grade AI products, APIs, RAG solutions and agentic workflows
Own complex AI initiatives from discovery and architecture through implementation, evaluation, deployment and ongoing improvement
Turn emerging AI capabilities into practical, reliable experiences with measurable value
Develop shared AI capabilities, reusable frameworks and self-service patterns for model orchestration, prompt management, agent execution and tool integration
Establish AI engineering standards and best practices for consistent, scalable delivery
Influence architecture and technical foundations for the broader AI ecosystem
Establish standards for AI testing, evaluation, observability, security, governance and production reliability
Build maintainable, scalable and resilient solutions
Improve performance, reliability and cost across AI systems
Partner with Product, Engineering, Data and Infrastructure teams
Provide architecture guidance, mentor engineers and contribute to technical decision-making
Share knowledge, document patterns and increase AI engineering capability across CreditorWatch
Deliver meaningful AI products or capabilities into production
Drive initiatives from problem definition through deployment and continuous improvement
Establish reusable frameworks, tooling and patterns for other teams
Build cross-functional relationships and influence technical decisions

Requirements

What you’ll need

Proven experience designing, building and operating production-grade AI systems
Experience in customer-facing or business-critical environments
Solid software engineering, system design and operational discipline
Ability to build maintainable and scalable solutions
Knowledge of LLM architecture, RAG, model orchestration, agentic workflows and tool integration
Experience with AI evaluation, observability, security and reliability in production
Experience with reusable frameworks, shared capabilities and developer experience
Ability to navigate ambiguity and make sensible technical trade-offs
Ownership of outcomes and ability to balance delivery speed with long-term quality
Ability to explain complex technical concepts clearly to technical and non-technical stakeholders
Experience with Langfuse, MLflow, OpenTelemetry and Datadog
Experience with PostgreSQL, OpenSearch and vector databases
Experience with AWS, Docker and GitHub
Experience designing and operating agentic or multi-agent systems in production
Practical experience with LLM evaluation, AI safety, governance and compliance controls
Experience building shared platforms, frameworks or developer capabilities adopted across multiple teams
Experience delivering technology in regulated or risk-sensitive environments
Track record of mentoring, providing architecture direction and driving cross-team engineering initiatives

Benefits

Comp & perks

Fitness First Platinum gym membership
Barista-made coffee, breakfast, snacks, lunches and drinks
$50 per month towards phone plan
Additional wellness day off each month
In-house monthly massages
Wine tasting, sailing, escape rooms, darts, boat days and team activities
Bring Your Family to Work Day
State-of-the-art office space in Sydney CBD
Professional development opportunities, new skills, challenges and career pathways
Employee Assistance Program with confidential support and resources
Inclusive workplace and equal opportunity employment