Head of AI

Kiss My Apps

. Design and build KMA’s internal AI operating system (AI OS) .

Posted 9/29/2026full-timeRemote • UkraineLeadWebsite

Core Competencies

Role fit

Core Competencies

Use this summary to align your resume positioning with the role.

Demonstrates expertise in building and launching AI/LLM systems, with a focus on production architecture, evaluation methodologies, and AI security. Proven ability to lead technical teams and influence stakeholders while driving automation and cost efficiency.

Highest-signal resume keywords

AI/LLM Systems DevelopmentProduction LLM ArchitectureAI Security KnowledgeTechnical Team ManagementExecutive Communication

ATS Keywords

Tailor your resume

Applicant Tracking System Keywords

Tip: use these terms in your resume and cover letter to boost ATS matches.

Hard Skills

AI Operating System DesignProduction LLM ArchitectureModel Gateway/Router DevelopmentEvaluation and Monitoring SystemsDrift DetectionData Pipeline UnderstandingCost Efficiency AnalysisBusiness Metrics DefinitionError AnalysisAutomation Impact Measurement

Soft Skills

Influence Without AuthorityStakeholder ManagementStrong Business Judgment

Tools & Technologies

LLM ProvidersData Quality ToolsAI Governance FrameworksHuman-in-the-Loop Controls

Industry Keywords

AI SecurityEU AI ActGDPRUnit EconomicsModel Routing

About the role

Key responsibilities & impact

Design and build KMA’s internal AI operating system (AI OS)
Develop production LLM architecture with model gateway/router, fallbacks, caching, cost caps, agent orchestration, and skill/agent registry
Develop the existing MCP Gateway and address its current gaps
Build offline and online evaluation and monitoring systems for AI solutions
Implement LLM-as-judge with calibration, pairwise comparison, judge-bias checks, failure taxonomies, drift detection, and degradation alerts
Configure LLM call tracing for tokens, latency, cache hits, and routing decisions, linking them to hallucination rate, groundedness, and task success
Build AI security protections including guardrails, tiered permissions, dangerous-access combination blocking, PII handling, secrets protection, and human-in-the-loop controls
Automate Ops, HR, and Finance processes with AI in partnership with the VP of Engineering
Increase the share of tasks AI can perform without developer involvement
Make model-routing and build-vs-buy-vs-fine-tune decisions based on cost and quality
Create methodology for measuring ROMI and LTV through holdout design, impact ledger, and baselines
Build and develop the AI Integration team
Participate in the hiring process through recruiter, technical, CTO, and final interviews

Requirements

What you’ll need

Experience building and launching AI/LLM systems in production, from problem framing through launch and impact measurement
Production LLM architecture experience: gateway/router, fallbacks, caching, rate limiting, observability, RAG, agents, and deployment patterns
Experience with multiple LLM providers and understanding trade-offs between API, managed inference, and self-hosted solutions
Experience building offline/online evaluations, LLM-as-judge, error analysis, failure taxonomies, and drift detection
Understanding of data pipelines, PII, data quality, and lineage
AI security knowledge: prompt injection, jailbreaks, data exfiltration, permissions, secrets, and human-in-the-loop
Understanding of AI governance, EU AI Act, and GDPR
Unit economics mindset: token costs, cost efficiency, and automation impact
Knowledge of build-vs-buy-vs-fine-tune and model routing by cost/quality
Experience defining business metrics, baselines, and measuring impact/ROMI
Hands-on approach and readiness to perform technical tasks independently until they can be delegated effectively
Influence without authority and stakeholder management outside direct reporting lines
Strong executive communication, business judgment, and data-driven approach
Experience building a small technical team
2–3+ years of hands-on AI/LLM production experience
At least 1 year of technical team management experience

Benefits

Comp & perks

Fully remote from any point in the free world, with optional access to offices in Kyiv and Warsaw
Equipment provided regardless of work format and geography
Career growth opportunities
Online yoga classes twice per week
English speaking clubs with a native speaker
Opportunities to speak at internal and external Kiss My Apps events
Annual budget covering up to 50% of professional conferences, educational events, and events
Fixed annual benefits budget allowing choice of 50% coverage for medical insurance, psychotherapy, sports, or dental care
40 days of paid vacation and sick leave without date restrictions
Support for uninterrupted work during wartime
Systematic support for the Defense Forces of Ukraine and participation in Ukraine recovery initiatives
Option for military reservation after the probationary period, subject to properly completed military registration documents and company capabilities under applicable law