Magic

Software Engineer

• Develop new benchmarks and evaluation methodologies
• Run data ablations
• Assess dataset quality
• Investigate whether measurements accurately reflect model performance
• Design and validate evaluations and benchmarks across model training and deployment
• Investigate model behavior and identify gaps in existing evaluations
• Identify opportunities to improve training data
• Improve evaluation correctness, reproducibility, and reliability
• Audit and improve public benchmarks, research papers, and open-source implementations
• Define metrics that accurately reflect model quality
• Build tools that make evaluation results accessible and useful across Magic

Explore Member of Technical Staff, Evals at Magic. Review requirements and application details.

Open roles

Member of Technical Staff, Evals