Senior Data Infrastructure Engineer – Scraping, Scale

AWISEE

. Build and maintain the core ingestion engine powering the social media data lab .

Posted 10/9/2026full-timeRemote • SwedenSeniorWebsite

Core Competencies

Role fit

Core Competencies

Use this summary to align your resume positioning with the role.

Expertise in building and maintaining scalable web-scraping pipelines and distributed systems, with a strong focus on data engineering and defeating anti-bot measures. Proficient in Python or Go, with experience in modern data lakes and real-time AI processing.

Highest-signal resume keywords

Web ScrapingPythonPlaywrightDistributed SystemsSQL

ATS Keywords

Tailor your resume

Applicant Tracking System Keywords

Tip: use these terms in your resume and cover letter to boost ATS matches.

Hard Skills

Web ScrapingData EngineeringReverse EngineeringPythonGoPlaywrightPuppeteerScrapySeleniumSQL

Tools & Technologies

CloudflareAkamaiPerimeterXTemporalRayKafkaRedis

Industry Keywords

Data LakesData WarehousesProxy ManagementAsynchronous TasksAI Modeling

Tech Stack

Tools & technologies

Distributed SystemsKafkaPuppeteerPythonRayRedisSeleniumSQLGo

About the role

Key responsibilities & impact

Build and maintain the core ingestion engine powering the social media data lab
Build and manage stealth scraping clusters using residential proxy networks, TLS fingerprinting, headful/headless browser farms, and session rotation
Build fault-tolerant, scalable web-scraping pipelines for Instagram, TikTok, YouTube, X, and web sources
Design distributed queues and workflow engines to manage millions of asynchronous scraping tasks daily
Architect structured and unstructured storage environments for downstream AI modeling
Implement automated alerts for platform UI/API changes, blocking patterns, and proxy failures
Deliver millions of profile, post, and video records daily with near-zero downtime

Requirements

What you’ll need

4+ years of experience in high-volume web scraping, data engineering, or reverse engineering
Deep experience defeating advanced anti-bot providers (Cloudflare, Akamai, PerimeterX) via TLS impersonation, browser automation, and proxy management
Mastery of Python or Go
Deep knowledge of Playwright, Puppeteer, Scrapy, or Selenium
Experience with distributed systems and task queues (Temporal, Ray, Kafka, Redis)
Strong SQL skills
Experience with modern analytical data lakes / data warehouses
Preferred: Direct experience extracting short-form video content and user metadata from TikTok, Instagram, and YouTube
Preferred: Experience integrating scraping outputs directly into vector databases and real-time AI processing queues

Benefits

Comp & perks

Remote work arrangement