Principal AI Security Engineer

Hewlett Packard Enterprise

. Design and develop scalable architectures for production-grade agentic AI systems .

Posted 11/5/2026full-timeUnited StatesLead💰 $126,500 - $290,500 per yearWebsite

Core Competencies

Role fit

Core Competencies

Use this summary to align your resume positioning with the role.

Demonstrates expertise in designing and developing scalable architectures for agentic AI systems, with strong capabilities in backend software engineering, distributed systems, and AI security principles. Proficient in implementing observability and evaluation mechanisms for agent behavior and system performance.

Highest-signal resume keywords

Agent OrchestrationDistributed SystemsBackend Software EngineeringAI Security PrinciplesMicroservices Architecture

ATS Keywords

Tailor your resume

Applicant Tracking System Keywords

Tip: use these terms in your resume and cover letter to boost ATS matches.

Hard Skills

PythonREST/gRPC APIsDockerKubernetesCI/CDMulti-Agent ArchitecturesModel EvaluationAsynchronous ServicesVector DatabasesHybrid Retrieval

Soft Skills

Clear Communication

Tools & Technologies

LangGraphClaude Agent SDKSemantic KernelAutoGenMCP Servers

Industry Keywords

Agentic AI SystemsScalabilityFault ToleranceObservabilitySecurity Engineering

Tech Stack

Tools & technologies

CloudDistributed SystemsDockerGRPCKubernetesMicroservicesPython

About the role

Key responsibilities & impact

Design and develop scalable architectures for production-grade agentic AI systems
Build agent orchestration, multi-agent workflows, state management, memory, tool integration, and human-in-the-loop capabilities
Design reusable execution harnesses and services separating AI reasoning and planning from deterministic execution
Develop model-agnostic interfaces supporting multiple foundation models, providers, and inference platforms
Build and integrate MCP servers, APIs, tool registries, skills, enterprise services, and data sources
Develop retrieval and context services including RAG, hybrid search, vector databases, and agent memory
Build backend services supporting asynchronous and distributed agent execution
Design evaluation and testing capabilities for agent behavior, tool usage, task completion, security, reliability, and performance
Implement observability across agent execution, model interactions, tools, state transitions, and system performance
Apply security engineering principles including least privilege, sandboxing, authorization, secrets management, guardrails, and auditability
Deploy and operate agentic services using containers, Kubernetes, CI/CD, and modern cloud-native engineering practices
Evaluate emerging models, frameworks, protocols, and technologies while maintaining architectural independence from individual vendors or platforms

Requirements

What you’ll need

Bachelor's or Master's degree in Computer Science, Engineering, Artificial Intelligence, or a related technical discipline, or equivalent practical experience
Typically 5–8 years of software engineering, platform engineering, distributed systems, AI engineering, or related experience
Demonstrated experience designing and delivering production software systems
Hands-on experience developing LLM-powered or agentic systems strongly preferred
Deep expertise in at least two of the five listed technical concentration areas and strong working knowledge across the remaining areas
Knowledge of distributed systems, microservices, APIs, event-driven and asynchronous architectures
Knowledge of workflow, state management, concurrency, scalability, and fault tolerance
Experience with agent orchestration, multi-agent architectures, planning, tool calling, delegation, state, memory, and long-running workflows
Experience with MCP, A2A, tool registries, skills, reusable agent capabilities, and execution harnesses
Experience with frameworks such as LangGraph, Claude Agent SDK, Semantic Kernel, AutoGen, or equivalent
Knowledge of foundation model APIs, open-weight/self-hosted models, model abstraction, routing, fallback, benchmarking, and workload-based model selection
Knowledge of structured generation, tool/function calling, RAG, hybrid retrieval, embeddings, re-ranking, vector databases, and context management
Strong Python and backend software engineering skills
Experience with REST/gRPC APIs, asynchronous services, messaging, and distributed processing
Experience with Docker, Kubernetes, CI/CD, and cloud infrastructure
Knowledge of logging, metrics, tracing, observability, production troubleshooting, performance, scalability, reliability, and resource optimization
Knowledge of agent and model evaluation, regression testing, task-success measurement, tool-use and trajectory evaluation
Knowledge of AI security, prompt injection, tool abuse, data leakage protections, least privilege, identity, authorization, sandboxing, and policy enforcement
Knowledge of guardrails, human approval gates, monitoring, traceability, and auditability
Ability to communicate system architecture, technical tradeoffs, and design decisions clearly

Benefits

Comp & perks

Comprehensive benefits supporting physical, financial, and emotional wellbeing
Personal and professional development programs
Flexible work arrangements to manage work and personal needs
Hybrid work arrangement
Inclusive and accessible workplace
Reasonable accommodation during the application or interview process for applicants with disabilities