Staff Software Engineer, Artificial Intelligence/LLM

Beacon Venture Capital

. Build user-facing LLM-powered product features end-to-end .

Posted 10/3/2026full-timeSan Carlos • California • United StatesLead💰 $224,000 - $260,000 per yearWebsite

Core Competencies

Role fit

Core Competencies

Use this summary to align your resume positioning with the role.

Demonstrates expertise in building LLM-powered product features, integrating with various model providers, and optimizing performance through effective data management and evaluation strategies. Proficient in Python or TypeScript, with a strong focus on technical leadership and collaboration across teams to ensure quality and reliability.

Highest-signal resume keywords

LLM Feature DevelopmentPython or TypeScript ProficiencyExperience with AWS Bedrock, OpenAI, AnthropicTechnical Leadership in Standards and DirectionData Evaluation and Success Metrics Design

ATS Keywords

Tailor your resume

Applicant Tracking System Keywords

Tip: use these terms in your resume and cover letter to boost ATS matches.

Hard Skills

Production CodingTesting and DocumentationEmbeddings and ChunkingVector Search TradeoffsAPIs and WorkersJSON/Schema-Bound OutputsCaching and Context PackingA/B Testing and RolloutsContent and Policy ChecksModel-Call Debugging

Soft Skills

Ability to Explain TradeoffsCollaboration with Cross-Functional TeamsProblem-Solving in Ambiguous Situations

Tools & Technologies

AWS BedrockOpenAIAnthropicOpenSearchPgvectorPineconeDynamoDBS3Dashboards and LogsModel Providers

Industry Keywords

User-Facing FeaturesRetrieval-Augmented GenerationFunction Calling FlowsLatency and Cost ManagementSensitive Data HandlingSecurity Standards Compliance

Tech Stack

Tools & technologies

AWSDynamoDBPythonTypeScript

About the role

Key responsibilities & impact

Build user-facing LLM-powered product features end-to-end
Design and implement retrieval-augmented generation and tool-calling flows
Deliver robust JSON and schema-bound outputs with validation, retries, and fallbacks
Add function calling to integrate internal tools, search, routing, and data services
Ship APIs and workers in Python or TypeScript with clear contracts, streaming, and backoff
Add caching, request shaping, prompt templates, and context packing to control latency and cost
Integrate with AWS Bedrock, OpenAI, Anthropic, or self-hosted endpoints
Collaborate with infrastructure teammates on chunking, embeddings, and indexing for documents, time series, and multimedia
Choose and tune vector backends such as OpenSearch, pgvector, or Pinecone
Maintain fresh knowledge bases through data synchronization from S3, Aurora, DynamoDB, and external sources
Create offline evaluations and golden sets for prompts, retrievers, and tools
Establish online metrics for task success, hallucination rate, retrieval precision/recall, p95 latency, and cost per request
Run A/B tests and prompt/version rollouts with guardrails and canaries
Implement content and policy checks, PII detection and redaction, access controls, and auditing
Design human-in-the-loop paths for sensitive actions
Handle aviation data according to internal security standards
Add tracing, logs, and dashboards for model calls, token usage, errors, and saturation
Debug failures across retrieval, prompts, tools, and providers
Set technical direction across services and teams for ambiguous, cross-team problems
Partner with ML/ infrastructure and product teammates on user experience, outcomes, reliability, and safety-critical systems

Requirements

What you’ll need

8+ years of experience, with a track record of owning systems or defining standards others build against
Alternatively, a master's degree with 4 years of experience related to the role
Experience shipping LLM features in front of users and improving them with data
Production coding, testing, and documentation skills
Understanding of embeddings, chunking, vector search tradeoffs, and function calling
Experience designing evaluations, defining success metrics, and iterating based on evidence
Ability to track p95 latency, meet SLAs, and reduce cost without hurting quality
Ability to explain tradeoffs and align product, infrastructure, and security partners
Technical leadership setting direction or standards used by other engineers or teams
Proficiency with Python or TypeScript
Experience with RAG, tool-calling flows, JSON/schema-bound outputs, validation, retries, fallbacks, APIs, workers, streaming, backoff, caching, prompt templates, context packing, and model providers
Experience with AWS Bedrock, OpenAI, Anthropic, or self-hosted endpoints
Knowledge of vector backends such as OpenSearch, pgvector, Pinecone, or Weaviate
Experience with data sources including S3, Aurora, and DynamoDB
Experience with offline evaluations, golden sets, online metrics, A/B testing, prompt/version rollouts, guardrails, and canaries
Experience implementing content and policy checks, PII detection and redaction, access controls, and auditing
Experience with tracing, logs, dashboards, model-call debugging, and production operations
U.S. Person status required; no visa sponsorship or visa transfers
All work must be performed in the United States
Ability to work in the San Francisco Bay Area / San Carlos, CA, with 3+ days per week onsite

Benefits

Comp & perks

Offers equity
100% of employee medical premiums covered
25% of dependent medical premiums covered
3 weeks PTO
13+ paid company holidays
401(k) offered
Remote work on remaining days of hybrid schedule