Novara

QA Engineer

• Design and maintain evaluation frameworks for LLM outputs and agentic workflows, including regression suites, golden datasets, and scoring rubrics
• Build test harnesses that catch hallucinations, tool-calling failures, prompt regressions, and unsafe or off-policy behavior before they reach production
• Define measurable quality criteria for agent reliability: task completion, factual grounding, latency, cost, and reasoning quality
• Integrate evaluation runs into CI/CD so model, prompt, and agent changes are gated the same way code changes are
• Partner with engineers on observability and tracing for agent runs, so failures are diagnosable rather than mysterious
• Contribute to conventional API and end-to-end automation where AI features sit inside larger product flows
• Help shape the team's shared playbook for testing AI features, and coach other QA engineers as agentic work spreads across scrum teams
• Work hands-on as an individual contributor reporting to the QA Manager
• Partner with product, engineering, and the broader QA team on LLM-driven and agentic workflows

Senior AI Quality Engineer

IT Support

• Provide support via phone, email, chat, ticket portal, and deskside, serving as the escalation point for tickets unresolved by first-line support
• Resolve incidents and service requests within established SLAs
• Triage, prioritize, and route tickets while keeping affected users informed
• Perform root-cause analysis on recurring issues and drive permanent fixes
• Provision, image, and deploy Windows and macOS endpoints using Autopilot, Intune, and Kandji
• Support Android and iOS devices through MDM
• Manage account onboarding and offboarding, Entra ID accounts, security groups, licenses, device provisioning, password resets, MFA resets, and SSO/SCIM administration
• Support Microsoft 365, including Exchange Online, SharePoint, OneDrive, mailboxes, permissions, shared mailboxes, distribution groups, and security groups
• Support Slack, Zoom, conference room AV equipment, and Zoom Rooms
• Support endpoint security using Microsoft Defender, CrowdStrike, BitLocker, FileVault, and device compliance tools
• Triage phishing and malware alerts and participate in incident response, remediation, and vulnerability management
• Troubleshoot LAN, Wi-Fi, VPN connectivity, and remote/hybrid access issues
• Maintain hardware and software asset inventories and license entitlements through procurement, deployment, refreshes, repairs, RMAs, and vendor support
• Create and maintain knowledge base articles, runbooks, SOPs, and technical documentation
• Develop and deliver end-user training and new-hire IT orientation
• Build and maintain automation scripts and workflows using PowerShell, Microsoft Graph API, and Power Automate
• Utilize AI-assisted support tools while validating recommended solutions
• Serve as a subject matter expert in technology domains and mentor other IT technicians
• Contribute to technology rollouts, system migrations, and hardware refresh projects

Senior AI Quality Engineer

IT Support Specialist