Research Scientist – Pre-training

Pluralis Research

. Build the training system that takes Protocol Learning from an 8B run to frontier scale .

Posted 10/14/2026full-timeRemote • California • United States, AustraliaMid-LevelSeniorWebsite

Core Competencies

Role fit

Core Competencies

Use this summary to align your resume positioning with the role.

Demonstrates expertise in distributed training systems, particularly in implementing model-parallel and data parallelism using frameworks like PyTorch. Proficient in building robust systems that ensure fault tolerance, monitoring, and efficient communication across heterogeneous hardware.

Highest-signal resume keywords

Distributed Training ExperienceModel-Parallel ImplementationProduction-Quality PythonData Parallelism UnderstandingConcurrency and Failure Handling

ATS Keywords

Tailor your resume

Applicant Tracking System Keywords

Tip: use these terms in your resume and cover letter to boost ATS matches.

Hard Skills

Distributed TrainingModel-Parallel TrainingData ParallelismTensor ParallelismPython ProgrammingProfiling and OptimizationCheckpointingState SynchronizationMonitoring SystemsReinforcement Learning

Soft Skills

Professional-Level English Proficiency

Tools & Technologies

PyTorchFSDPDeepSpeedMegatronP2P Networking

Industry Keywords

Protocol LearningLarge Language ModelsNAT TraversalInference SystemsOpen-Source AI Labs

Tech Stack

Tools & technologies

Node.jsPythonPyTorch

About the role

Key responsibilities & impact

Build the training system that takes Protocol Learning from an 8B run to frontier scale
Implement and optimize model-parallel distributed pretraining across heterogeneous GPUs
Implement data, pipeline, and tensor parallelism over low-bandwidth, high-latency links
Reduce communication overhead while maintaining model convergence
Build elasticity and fault tolerance so runs survive node churn
Implement robust checkpointing, state synchronization, and recovery as participants join and leave
Build monitoring to show throughput, bottlenecks, and model quality across hundreds of devices
Train and serve large models on heterogeneous hardware in physically different regions over ordinary internet connections

Requirements

What you’ll need

Hands-on distributed training required: trained models across many devices in PyTorch using FSDP, DeepSpeed, Megatron, or a custom implementation
Understanding of data, tensor, and pipeline parallelism
Production-quality Python
Concurrency and failure handling experience
Profiling before optimizing
Evidence of execution through shipped systems, research code, open-source work, or serious personal projects
Mission alignment with Protocol Learning
Professional-level English proficiency, written and spoken
Nice to have: experience training or serving large language models such as Nemotron, Qwen, or OLMo
Nice to have: P2P networking and NAT traversal experience
Nice to have: post-training and reinforcement learning experience
Nice to have: inference and serving systems experience
Nice to have: experience at proprietary, open-weight, or open-source AI labs

Benefits

Comp & perks

Significant ownership/equity for key technical contributors
High base salary
Flexible work environment
Full visa sponsorship and relocation support to either Australia or the US for exceptional candidates
Opportunity to work on open, unsolved research problems
Remote-first culture
Globally distributed team