AI Operations Specialist

Maleda Tech

. Prioritize projects by business impact and make the case for worthwhile initiatives .

Posted 10/27/2026contractRemote • United StatesMid-LevelSenior💰 $65 - $75 per hourWebsite

Core Competencies

Role fit

Core Competencies

Use this summary to align your resume positioning with the role.

Demonstrates expertise in building and managing AI workflows, ensuring quality and accuracy through metrics and guidelines. Proven ability to collaborate with cross-functional teams and drive measurable impact in operations and program management.

Highest-signal resume keywords

AI Workflow DevelopmentSQL ExperiencePython ProgrammingQuality AssuranceProcess Measurement

ATS Keywords

Tailor your resume

Applicant Tracking System Keywords

Tip: use these terms in your resume and cover letter to boost ATS matches.

Hard Skills

AI WorkflowsPrompt WritingPipeline SetupAccuracy MetricsRecall MetricsSampling MetricsLabeling GuidelinesHuman-Verified LabelsVendor ManagementModel Calibration

Soft Skills

Clear WritingSelf-DirectionCollaboration

Tools & Technologies

APIs

Industry Keywords

Operations ManagementProgram ManagementApplied AIContent ModerationHuman-in-the-LoopTwo-Sided Marketplace

Tech Stack

Tools & technologies

PythonSQL

About the role

Key responsibilities & impact

Prioritize projects by business impact and make the case for worthwhile initiatives
Map current workflows with process owners and define where AI and human intervention should be used
Build AI workflows, write prompts, set up pipelines, maintain quality levels, and monitor post-launch drift
Build test sets, measure accuracy, and calibrate LLM judges against human reviewers
Define launch-readiness criteria for AI features and provide evidence for go/no-go decisions
Turn unclear policy into consistently applicable review criteria with product, legal, and policy partners
Create labeling guidelines and produce human-verified labels on real data
Run labeling at scale with vendor agents and validate work using agreement metrics and spot checks
Document reusable solutions and learnings
Keep methods current as models, tools, and vendor agents change, retiring ineffective methods
Own assigned workstreams end to end, from setup through quality checks

Requirements

What you’ll need

5–7 years in operations, program management, analytics, consulting, or applied AI, with accountability for results
Daily AI user who has built something with AI that delivered measurable impact, such as a workflow, automation, or analysis
Comfortable with accuracy, precision, recall, and sampling metrics
Experience making a messy process measurable
Quality experience, such as review guidelines, QA, or audits
Clear writing and ability to work with business leads, product managers, and engineers
Self-directed and comfortable with limited direction
Must be authorized to work in the U.S. without sponsorship
Preferred: hands-on SQL, Python, or API experience
Preferred: trust and safety, content moderation, annotation operations, or other human-in-the-loop work at scale
Preferred: experience fine-tuning models or calibrating LLM judges
Preferred: background in a two-sided marketplace, travel, or another operations-heavy business
Preferred: experience setting up intake and prioritization for a new team

Benefits

Comp & perks

Benefits (unspecified)