P-1 AI

Software Engineer

• Study how engineers and customers use Archie in the field, identify where Archie succeeds or fails from an engineering perspective, and turn those observations into actionable evaluations
• Recreate real-world engineering tasks and failure modes in environments that can be repeatedly used by development teams
• Build and refine evaluation methodologies for Archie, including automated judges and human evaluation processes
• Assess whether automated judges align with expert human judgment and improve them when they do not
• Work across engineering, product, customers, subject matter experts, and external partners to ensure evaluations reflect real engineering workflows rather than abstract benchmarks
• Use evaluation results to help the team understand where Archie needs to improve and which problems matter most to users

Explore Member of Technical Staff, AI Evaluation at P-1 AI. Review requirements and application details.