FREE ACCESS
5,000–10,000 jobs/day
See all jobs on Scoutfield
Search thousands of fresh jobs every day.
Discover
- Fresh listings
- Fast filters
- No subscription required
Create a free account and start exploring right away.
Core Competencies
Role fitCore Competencies
Use this summary to align your resume positioning with the role.
Demonstrates expertise in building and scaling large-scale data systems and pipelines for machine learning, with a focus on dataset curation, filtering, and quality improvement. Proficient in cloud-based data storage and processing, container orchestration, and implementing best practices for data management.
Highest-signal resume keywords
Large-Scale Data SystemsDistributed Data ProcessingContainer OrchestrationCloud-Based Data StorageVLM-Based Captioning
ATS Keywords
Tailor your resumeApplicant Tracking System Keywords
Tip: use these terms in your resume and cover letter to boost ATS matches.
Hard Skills
PythonDataset CurationData FilteringQuality ImprovementData Pipeline DesignOrchestration ToolsSemantic Data SelectionVideo ProcessingMachine LearningData Analysis
Soft Skills
Problem-SolvingCommunicationDocumentation
Tools & Technologies
KubernetesDockerAWSGCSAzurePySparkRayAirflowFFmpegOpenCV
Industry Keywords
Data Processing FrameworksContainerizationCuration PipelinesQuality/Aesthetic Scoring ModelsEmbedding-Based Filtering
Tech Stack
Tools & technologiesAirflowAWSAzureCloudDockerFFmpegKubernetesPySparkPythonRay
About the role
Key responsibilities & impact- Build and scale systems for ingesting, processing, and delivering large-scale video and multimodal data for model training
- Own the full pipeline from raw content to curated, filtered, and training-ready datasets
- Design and scale distributed data pipelines for preprocessing, dataset generation, and repeated dataset refreshes
- Own workflow orchestration, job scheduling, monitoring, and failure recovery for large-scale data processing jobs
- Implement and maintain containerized pipeline infrastructure using Kubernetes or equivalent orchestration systems
- Optimize cloud-based data storage and movement across AWS, GCS, or Azure for cost, throughput, and operational efficiency
- Define and implement best practices for dataset storage layout, versioning, caching, retention, and access patterns
- Design and implement curation pipelines for selecting, filtering, and retaining video and image content for model training
- Build and improve VLM-based captioning and metadata generation workflows at scale across video and image data
- Develop and apply quality and aesthetic scoring models, CLIP-based semantic filtering, and other signal-extraction approaches for data selection
- Build tooling for near-deduplication and exact deduplication pipelines over large video corpora
- Analyze dataset composition, identify quality issues, and iterate on curation logic to improve training outcomes
- Define and evolve standards for high-quality, training-ready video data across different training regimes
- Partner closely with curation and modeling teams to operationalize evolving dataset recipes and improve model outcomes
Requirements
What you’ll need- Strong hands-on experience building or scaling large-scale data systems and pipelines for machine learning, including dataset curation, filtering, and quality improvement
- Experience with distributed data processing frameworks such as PySpark or Ray, and orchestration tools such as Airflow or equivalent
- Familiarity with containerization and container orchestration, including Docker and Kubernetes
- Experience working with cloud-based data storage and compute (AWS, GCS, and/or Azure), including tradeoffs around cost, throughput, storage layout, and access patterns
- Experience with VLM-based captioning pipelines or quality/aesthetic scoring models for video or image data, including curation of image-text pair datasets for joint image-video training
- Familiarity with CLIP-based or embedding-based filtering and semantic data selection techniques
- Familiarity with video and media processing tools such as FFmpeg, PyAV, DALI, or OpenCV, and relevant libraries such as Decord, torchvision, PyTorchVideo, or torchaudio
- Proficiency in Python
- Strong problem-solving, communication, and documentation skills
Benefits
Comp & perks- Competitive salary and generous company equity
- Personal time off and paid holidays
- Health insurance
- Global travel insurance: Covers you when traveling internationally
- Monthly spending stipend: $500 (~S$635)
- Equipment: All equipment needed for your home office
